Jul 06 20:23:18.597851 ip-10-0-128-47 systemd[1]: Starting Kubernetes Kubelet...
Jul 06 20:23:19.005074 ip-10-0-128-47 kubenswrapper[2554]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:23:19.005074 ip-10-0-128-47 kubenswrapper[2554]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 06 20:23:19.005074 ip-10-0-128-47 kubenswrapper[2554]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:23:19.005074 ip-10-0-128-47 kubenswrapper[2554]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 06 20:23:19.005074 ip-10-0-128-47 kubenswrapper[2554]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 06 20:23:19.006273 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.006191    2554 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 06 20:23:19.011208 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011191    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.011208 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011207    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011211    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011214    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011218    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011222    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011225    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011228    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011231    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011234    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011237    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011239    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011242    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011245    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011248    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011251    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011253    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011256    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011258    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011261    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.011272 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011263    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011266    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011269    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011271    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011274    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011277    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011280    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011283    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011286    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011288    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011291    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011294    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011298    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011302    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011305    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011307    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011310    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011313    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011315    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.011751 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011318    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011320    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011323    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011325    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011328    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011331    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011334    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011336    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011339    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011341    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011344    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011346    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011349    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011351    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011354    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011358    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011361    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011363    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011366    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011369    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.012202 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011371    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011374    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011376    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011378    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011381    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011384    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011386    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011389    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011391    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011394    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011397    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011401    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011404    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011406    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011411    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011414    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011417    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011420    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011423    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011426    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.012703 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011429    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011432    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011434    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011437    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011439    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011442    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011445    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011448    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011451    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011453    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011456    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011458    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011461    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011465    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011467    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011470    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011474    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011477    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011480    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.013170 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.011483    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013117    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013124    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013128    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013131    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013134    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013137    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013139    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013142    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013145    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013154    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013156    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013160    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013162    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013165    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013169    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013171    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013174    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013177    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.013640 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013180    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013183    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013186    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013189    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013192    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013194    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013197    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013200    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013202    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013205    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013207    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013210    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013212    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013215    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013217    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013220    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013223    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013225    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013228    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013231    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.014139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013233    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013236    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013239    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013242    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013244    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013247    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013250    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013252    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013255    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013258    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013261    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013263    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013266    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013269    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013271    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013274    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013276    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013279    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013282    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013284    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.014634 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013287    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013289    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013292    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013294    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013297    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013299    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013302    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013305    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013307    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013309    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013312    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013314    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013317    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013319    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013322    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013325    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013327    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013330    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013333    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013339    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.015118 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013343    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013346    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013349    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013352    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013354    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013357    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013359    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013362    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013364    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013367    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013369    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013372    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013374    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013377    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013379    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013382    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013385    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013387    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013390    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.015616 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013392    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013396    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013790    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013796    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013799    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013802    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013804    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013807    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013809    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013812    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013814    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013817    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013820    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013823    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013826    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013828    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013831    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013834    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013837    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.016076 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013839    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013842    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013844    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013847    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013850    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013855    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013858    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013860    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013863    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013865    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013868    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013871    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013873    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013876    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013878    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013881    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013883    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013886    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013888    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013891    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.016555 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013893    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013896    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013898    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013901    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013903    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013906    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013909    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013914    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013917    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013920    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013923    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013925    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013928    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013931    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013933    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013937    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013940    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013944    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013946    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.017048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013948    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013951    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013953    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013956    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013959    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013961    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013963    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013966    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013969    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013971    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013973    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013976    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013979    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013982    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013985    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013987    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013990    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013992    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013995    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.013997    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.017510 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014000    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014003    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014005    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014008    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014010    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014012    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014015    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014018    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014020    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014023    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014029    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014032    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014035    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014039    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014042    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014045    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014047    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014050    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014054    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014057    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.017996 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014059    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014063    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.014066    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014136    2554 flags.go:64] FLAG: --address="0.0.0.0"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014143    2554 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014150    2554 flags.go:64] FLAG: --anonymous-auth="true"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014157    2554 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014163    2554 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014168    2554 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014174    2554 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014181    2554 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014185    2554 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014188    2554 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014193    2554 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014196    2554 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014199    2554 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014202    2554 flags.go:64] FLAG: --cgroup-root=""
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014205    2554 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014208    2554 flags.go:64] FLAG: --client-ca-file=""
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014211    2554 flags.go:64] FLAG: --cloud-provider="external"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014214    2554 flags.go:64] FLAG: --cluster-dns="[]"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014218    2554 flags.go:64] FLAG: --cluster-domain=""
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014221    2554 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014224    2554 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 06 20:23:19.018593 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014227    2554 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014231    2554 flags.go:64] FLAG: --container-log-max-files="5"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014235    2554 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014238    2554 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014241    2554 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014244    2554 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014247    2554 flags.go:64] FLAG: --contention-profiling="false"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014250    2554 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014253    2554 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014256    2554 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014259    2554 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014264    2554 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014267    2554 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014270    2554 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014273    2554 flags.go:64] FLAG: --enable-load-reader="false"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014276    2554 flags.go:64] FLAG: --enable-server="true"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014278    2554 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014288    2554 flags.go:64] FLAG: --event-burst="100"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014291    2554 flags.go:64] FLAG: --event-qps="50"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014294    2554 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014297    2554 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014300    2554 flags.go:64] FLAG: --eviction-hard=""
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014304    2554 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014307    2554 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014310    2554 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 06 20:23:19.019194 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014313    2554 flags.go:64] FLAG: --eviction-soft=""
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014316    2554 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014319    2554 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014322    2554 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014325    2554 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014328    2554 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014331    2554 flags.go:64] FLAG: --fail-swap-on="true"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014334    2554 flags.go:64] FLAG: --feature-gates=""
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014338    2554 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014341    2554 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014344    2554 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014347    2554 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014350    2554 flags.go:64] FLAG: --healthz-port="10248"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014353    2554 flags.go:64] FLAG: --help="false"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014356    2554 flags.go:64] FLAG: --hostname-override="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014359    2554 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014362    2554 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014364    2554 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014371    2554 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014374    2554 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014377    2554 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014380    2554 flags.go:64] FLAG: --image-service-endpoint=""
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014383    2554 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 06 20:23:19.019816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014386    2554 flags.go:64] FLAG: --kube-api-burst="100"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014389    2554 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014392    2554 flags.go:64] FLAG: --kube-api-qps="50"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014395    2554 flags.go:64] FLAG: --kube-reserved=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014398    2554 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014401    2554 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014404    2554 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014407    2554 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014410    2554 flags.go:64] FLAG: --lock-file=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014413    2554 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014416    2554 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014419    2554 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014424    2554 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014427    2554 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014430    2554 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014432    2554 flags.go:64] FLAG: --logging-format="text"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014435    2554 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014439    2554 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014441    2554 flags.go:64] FLAG: --manifest-url=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014444    2554 flags.go:64] FLAG: --manifest-url-header=""
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014449    2554 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014452    2554 flags.go:64] FLAG: --max-open-files="1000000"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014456    2554 flags.go:64] FLAG: --max-pods="110"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014459    2554 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014462    2554 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 06 20:23:19.020411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014465    2554 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014468    2554 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014472    2554 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014475    2554 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014478    2554 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014485    2554 flags.go:64] FLAG: --node-status-max-images="50"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014488    2554 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014491    2554 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014495    2554 flags.go:64] FLAG: --pod-cidr=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014498    2554 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014503    2554 flags.go:64] FLAG: --pod-manifest-path=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014506    2554 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014509    2554 flags.go:64] FLAG: --pods-per-core="0"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014512    2554 flags.go:64] FLAG: --port="10250"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014515    2554 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014518    2554 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-060596e39eb70c09a"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014521    2554 flags.go:64] FLAG: --qos-reserved=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014524    2554 flags.go:64] FLAG: --read-only-port="10255"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014547    2554 flags.go:64] FLAG: --register-node="true"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014560    2554 flags.go:64] FLAG: --register-with-taints=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014564    2554 flags.go:64] FLAG: --registry-burst="10"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014567    2554 flags.go:64] FLAG: --registry-qps="5"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014570    2554 flags.go:64] FLAG: --reserved-cpus=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014573    2554 flags.go:64] FLAG: --reserved-memory=""
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014577    2554 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 06 20:23:19.021076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014580    2554 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014583    2554 flags.go:64] FLAG: --rotate-certificates="false"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014586    2554 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014588    2554 flags.go:64] FLAG: --runonce="false"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014591    2554 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014594    2554 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014597    2554 flags.go:64] FLAG: --seccomp-default="false"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014600    2554 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014603    2554 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014606    2554 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014611    2554 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014614    2554 flags.go:64] FLAG: --storage-driver-password="root"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014616    2554 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014619    2554 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014622    2554 flags.go:64] FLAG: --storage-driver-user="root"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014626    2554 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014629    2554 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014632    2554 flags.go:64] FLAG: --system-cgroups=""
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014635    2554 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014641    2554 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014644    2554 flags.go:64] FLAG: --tls-cert-file=""
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014647    2554 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014651    2554 flags.go:64] FLAG: --tls-min-version=""
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014654    2554 flags.go:64] FLAG: --tls-private-key-file=""
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014657    2554 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 06 20:23:19.021722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014660    2554 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014663    2554 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014666    2554 flags.go:64] FLAG: --v="2"
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014670    2554 flags.go:64] FLAG: --version="false"
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014674    2554 flags.go:64] FLAG: --vmodule=""
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014678    2554 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.014681    2554 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015150    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015155    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015158    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015161    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015164    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015167    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015169    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015172    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015175    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015177    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015182    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015184    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015187    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015189    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.022329 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015192    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015196    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015200    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015202    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015205    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015208    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015211    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015214    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015216    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015219    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015222    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015225    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015227    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015230    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015233    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015235    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015238    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015241    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015243    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015246    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.022956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015249    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015251    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015254    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015256    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015259    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015261    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015264    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015266    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015270    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015273    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015275    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015278    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015280    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015284    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015287    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015289    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015292    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015295    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015297    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.023434 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015299    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015302    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015305    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015307    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015310    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015313    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015315    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015318    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015320    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015323    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015325    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015328    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015330    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015333    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015335    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015338    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015340    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015343    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015345    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015348    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.023916 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015350    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015354    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015356    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015359    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015362    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015364    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015368    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015370    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015373    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015376    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015378    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015381    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015383    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015387    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015391    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015394    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015396    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015399    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015403    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015406    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.024387 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015409    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015412    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015414    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015417    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015421    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.015424    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.015430    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022038    2554 server.go:529] "Kubelet version" kubeletVersion="v1.34.8"
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022053    2554 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022073    2554 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022079    2554 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022135    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022140    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022143    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022147    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.024876 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022150    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022152    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022155    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022158    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022161    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022164    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022166    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022170    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022174    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022177    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022181    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022184    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022187    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022189    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022193    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022196    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022199    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022202    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022204    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.025286 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022207    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022210    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022212    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022215    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022218    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022220    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022224    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022227    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022230    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022232    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022235    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022237    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022240    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022242    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022245    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022248    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022251    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022253    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022256    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022258    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.025758 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022261    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022264    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022266    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022269    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022271    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022274    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022277    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022279    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022281    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022284    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022286    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022289    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022291    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022294    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022296    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022299    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022301    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022304    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022307    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022310    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.026244 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022312    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022315    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022317    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022320    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022322    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022325    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022327    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022330    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022332    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022335    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022337    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022340    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022342    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022345    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022348    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022350    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022352    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022355    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022358    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022360    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.026828 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022363    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022365    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022368    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022372    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022375    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022378    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022381    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022383    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022386    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022388    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022391    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022394    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022397    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022401    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022404    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.027304 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022407    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022412    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022522    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022542    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022546    2554 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022549    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022552    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022556    2554 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022560    2554 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022562    2554 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022565    2554 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022567    2554 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022570    2554 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022573    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022575    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 06 20:23:19.027699 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022578    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022580    2554 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022583    2554 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022585    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022588    2554 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022591    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022593    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022596    2554 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022598    2554 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022601    2554 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022604    2554 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022606    2554 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022609    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022611    2554 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022614    2554 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022617    2554 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022619    2554 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022622    2554 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022624    2554 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 06 20:23:19.028071 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022627    2554 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022629    2554 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022631    2554 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022634    2554 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022637    2554 feature_gate.go:328] unrecognized feature gate: Example
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022639    2554 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022642    2554 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022644    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022647    2554 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022649    2554 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022651    2554 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022654    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022656    2554 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022659    2554 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022661    2554 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022664    2554 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022666    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022669    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022671    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022674    2554 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 06 20:23:19.028545 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022676    2554 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022679    2554 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022681    2554 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022684    2554 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022686    2554 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022689    2554 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022691    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022693    2554 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022696    2554 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022699    2554 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022701    2554 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022705    2554 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022708    2554 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022711    2554 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022714    2554 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022719    2554 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022722    2554 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022725    2554 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022728    2554 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022730    2554 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 06 20:23:19.029032 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022733    2554 feature_gate.go:328] unrecognized feature gate: Example2
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022736    2554 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022739    2554 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022741    2554 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022744    2554 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022746    2554 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022749    2554 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022752    2554 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022754    2554 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022757    2554 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022759    2554 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022762    2554 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022764    2554 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022767    2554 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022769    2554 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022772    2554 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022774    2554 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022777    2554 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022779    2554 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022782    2554 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 06 20:23:19.029541 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022784    2554 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022787    2554 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022789    2554 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022792    2554 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022796    2554 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022799    2554 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:19.022802    2554 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022810    2554 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.022913    2554 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.024953    2554 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.026451    2554 server.go:1019] "Starting client certificate rotation"
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.026557    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.026588    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 06 20:23:19.030043 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.029372    2554 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 20:23:19.030840 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.030803    2554 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 06 20:23:19.040833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.040813    2554 log.go:25] "Validated CRI v1 runtime API"
Jul 06 20:23:19.042745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.042728    2554 log.go:25] "Validated CRI v1 image API"
Jul 06 20:23:19.045287 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.045268    2554 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 06 20:23:19.047829 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.047810    2554 fs.go:135] Filesystem UUIDs: map[098c765a-0e63-449e-bc6e-b8b9408f60eb:/dev/nvme0n1p3 648bb8bb-e09c-47aa-9db0-d281b8fd6c5e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 06 20:23:19.047887 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.047828    2554 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 06 20:23:19.053771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.053656    2554 manager.go:216] Machine: {Timestamp:2026-07-06 20:23:19.052330069 +0000 UTC m=+0.336380601 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101154 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec208822dffa47586f128f1b3f3bacd1 SystemUUID:ec208822-dffa-4758-6f12-8f1b3f3bacd1 BootID:054525ec-1884-4d1d-b175-129dfe5bc7c3 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:f9:06:c9:79:73 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:f9:06:c9:79:73 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:1b:91:7b:7b:56 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 06 20:23:19.053771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.053763    2554 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 06 20:23:19.053943 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.053839    2554 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 06 20:23:19.056995 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.056969    2554 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 06 20:23:19.057165 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.057002    2554 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-47.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 06 20:23:19.058376 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.058364    2554 topology_manager.go:138] "Creating topology manager with none policy"
Jul 06 20:23:19.058440 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.058382    2554 container_manager_linux.go:309] "Creating device plugin manager"
Jul 06 20:23:19.058440 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.058399    2554 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 20:23:19.058666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.058650    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 20:23:19.059066 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.059052    2554 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 06 20:23:19.060452 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.060440    2554 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 06 20:23:19.063489 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063477    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 20:23:19.063658 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063647    2554 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 06 20:23:19.063726 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063693    2554 kubelet.go:486] "Attempting to sync node with API server"
Jul 06 20:23:19.063726 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063708    2554 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 06 20:23:19.063820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063728    2554 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 06 20:23:19.063820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063742    2554 kubelet.go:398] "Adding apiserver pod source"
Jul 06 20:23:19.063820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.063755    2554 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 06 20:23:19.064181 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.064161    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 20:23:19.064247 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.064184    2554 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 06 20:23:19.066516 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.066493    2554 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1"
Jul 06 20:23:19.068866 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.068853    2554 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 06 20:23:19.068918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.068884    2554 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 06 20:23:19.070073 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070061    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070078    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070084    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070089    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070098    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070103    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070111    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070116    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070123    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 06 20:23:19.070127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070129    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 06 20:23:19.070386 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070147    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 06 20:23:19.070386 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070156    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 06 20:23:19.070946 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070936    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 06 20:23:19.070946 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.070945    2554 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 06 20:23:19.074314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.074299    2554 server.go:1268] "Started kubelet"
Jul 06 20:23:19.074407 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.074387    2554 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 06 20:23:19.074486 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.074406    2554 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 06 20:23:19.074486 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.074458    2554 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 06 20:23:19.074807 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.074787    2554 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-47.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 06 20:23:19.074905 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.074855    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 06 20:23:19.074972 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.074949    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 06 20:23:19.075248 ip-10-0-128-47 systemd[1]: Started Kubernetes Kubelet.
Jul 06 20:23:19.079761 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.079605    2554 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 06 20:23:19.083724 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.083708    2554 server.go:310] "Adding debug handlers to kubelet server"
Jul 06 20:23:19.083916 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.083898    2554 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 06 20:23:19.085602 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.084732    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18bfcd050f1fa964  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-07-06 20:23:19.07426954 +0000 UTC m=+0.358320073,LastTimestamp:2026-07-06 20:23:19.07426954 +0000 UTC m=+0.358320073,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}"
Jul 06 20:23:19.089040 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089026    2554 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 06 20:23:19.089105 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089032    2554 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 06 20:23:19.089471 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.089215    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.089658 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089638    2554 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 06 20:23:19.089658 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089658    2554 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 06 20:23:19.089817 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089800    2554 reconstruct.go:97] "Volume reconstruction finished"
Jul 06 20:23:19.089817 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089814    2554 reconciler.go:29] "Reconciler: start to sync state"
Jul 06 20:23:19.089817 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.089816    2554 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 06 20:23:19.090067 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090049    2554 factory.go:55] Registering systemd factory
Jul 06 20:23:19.090620 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090607    2554 factory.go:223] Registration of the systemd container factory successfully
Jul 06 20:23:19.090849 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090835    2554 factory.go:153] Registering CRI-O factory
Jul 06 20:23:19.090896 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090854    2554 factory.go:223] Registration of the crio container factory successfully
Jul 06 20:23:19.090929 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090911    2554 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 06 20:23:19.090962 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090939    2554 factory.go:103] Registering Raw factory
Jul 06 20:23:19.090962 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.090953    2554 manager.go:1196] Started watching for new ooms in manager
Jul 06 20:23:19.091400 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.091386    2554 manager.go:349] Starting recovery of all containers
Jul 06 20:23:19.098837 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.098808    2554 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 06 20:23:19.099379 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.099347    2554 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-47.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 06 20:23:19.102441 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.102425    2554 manager.go:354] Recovery completed
Jul 06 20:23:19.107419 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.107407    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.109384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.109367    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.109473 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.109403    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.109473 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.109418    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.110451 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.110438    2554 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 06 20:23:19.110451 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.110450    2554 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 06 20:23:19.110595 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.110471    2554 state_mem.go:36] "Initialized new in-memory state store"
Jul 06 20:23:19.110595 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.110451    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.111317 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.111304    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.111381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.111331    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.111381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.111357    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.111740 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.111629    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18bfcd05113779a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-07-06 20:23:19.109384616 +0000 UTC m=+0.393435155,LastTimestamp:2026-07-06 20:23:19.109384616 +0000 UTC m=+0.393435155,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}"
Jul 06 20:23:19.115901 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.115878    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8r2h4"
Jul 06 20:23:19.122859 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.122792    2554 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-47.ec2.internal.18bfcd051137dcaf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-47.ec2.internal,UID:ip-10-0-128-47.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-47.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-47.ec2.internal,},FirstTimestamp:2026-07-06 20:23:19.109409967 +0000 UTC m=+0.393460503,LastTimestamp:2026-07-06 20:23:19.109409967 +0000 UTC m=+0.393460503,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-47.ec2.internal,}"
Jul 06 20:23:19.123160 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.123144    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8r2h4"
Jul 06 20:23:19.190349 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.190320    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.251951 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.251927    2554 policy_none.go:49] "None policy: Start"
Jul 06 20:23:19.252014 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.251956    2554 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 06 20:23:19.252014 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.251971    2554 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 06 20:23:19.253090 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.253072    2554 policy_none.go:47] "Start"
Jul 06 20:23:19.274475 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.273544    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 06 20:23:19.275077 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.275058    2554 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 06 20:23:19.275158 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.275106    2554 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 06 20:23:19.275158 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.275136    2554 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 06 20:23:19.275234 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.275176    2554 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]"
Jul 06 20:23:19.282078 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.282063    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:19.290093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290079    2554 manager.go:337] "Starting Device Plugin manager"
Jul 06 20:23:19.290176 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.290112    2554 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 06 20:23:19.290176 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290122    2554 server.go:87] "Starting device plugin registration server"
Jul 06 20:23:19.290396 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290378    2554 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 06 20:23:19.290492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290393    2554 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 06 20:23:19.290492 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.290413    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.290492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290473    2554 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 06 20:23:19.290703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290688    2554 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 06 20:23:19.290703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.290701    2554 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 06 20:23:19.291631 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.291615    2554 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 06 20:23:19.291713 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.291654    2554 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.305174 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.305157    2554 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-47.ec2.internal\" not found" node="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.375467 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.375430    2554 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"]
Jul 06 20:23:19.375623 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.375558    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.376561 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.376540    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.376629 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.376577    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.376629 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.376595    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.378181 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378167    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.378302 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378288    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.378339 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378331    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.378945 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378920    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.379046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378955    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.379046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378970    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.379046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.378924    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.379046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.379016    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.379046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.379027    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.380258 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.380241    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.380338 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.380274    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.380993 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.380978    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.381088 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.381005    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.381088 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.381018    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.390460 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.390446    2554 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 06 20:23:19.390558 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.390496    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.390558 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.390516    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.390558 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.390545    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61ef1a50927e5ff318f1211f698ffaa1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"61ef1a50927e5ff318f1211f698ffaa1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.391109 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.391097    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientMemory"
Jul 06 20:23:19.391154 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.391124    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasNoDiskPressure"
Jul 06 20:23:19.391154 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.391136    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeHasSufficientPID"
Jul 06 20:23:19.391208 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.391158    2554 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.399686 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.399669    2554 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.399747 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.399687    2554 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-47.ec2.internal\": node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.404380 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.404355    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-47.ec2.internal\" not found" node="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.408793 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.408779    2554 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-47.ec2.internal\" not found" node="ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.413486 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.413472    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.491117 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491085    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.491117 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491120    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.491312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491147    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61ef1a50927e5ff318f1211f698ffaa1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"61ef1a50927e5ff318f1211f698ffaa1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.491312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491193    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.491312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491223    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/61ef1a50927e5ff318f1211f698ffaa1-config\") pod \"kube-apiserver-proxy-ip-10-0-128-47.ec2.internal\" (UID: \"61ef1a50927e5ff318f1211f698ffaa1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.491312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.491197    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/51e41b390cac1ae2594606d5b623e99b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal\" (UID: \"51e41b390cac1ae2594606d5b623e99b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.513997 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.513962    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.614870 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.614785    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.707312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.707274    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.712799 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.712781    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:19.714859 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.714844    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.815069 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.815032    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.915900 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:19.915832    2554 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-47.ec2.internal\" not found"
Jul 06 20:23:19.926486 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.926467    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:19.993970 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:19.993930    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:20.009784 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.009764    2554 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 06 20:23:20.012977 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.012961    2554 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:20.026207 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.026191    2554 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 06 20:23:20.026345 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.026326    2554 request.go:1196] "Unexpected error when reading response body" err="read tcp 10.0.128.47:44684->3.214.175.228:6443: use of closed network connection"
Jul 06 20:23:20.026404 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.026359    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 20:23:20.026404 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.026368    2554 kubelet.go:3250] "Failed creating a mirror pod" err="unexpected error when reading response body. Please retry. Original error: read tcp 10.0.128.47:44684->3.214.175.228:6443: use of closed network connection" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal"
Jul 06 20:23:20.026404 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.026367    2554 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 06 20:23:20.063952 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.063927    2554 apiserver.go:52] "Watching apiserver"
Jul 06 20:23:20.068248 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.068231    2554 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:20.073011 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.072994    2554 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 06 20:23:20.075702 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.075682    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-zb467","openshift-multus/network-metrics-daemon-k74c5","openshift-ovn-kubernetes/ovnkube-node-59hxf","kube-system/global-pull-secret-syncer-xr6rj","kube-system/konnectivity-agent-x57bd","kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal","openshift-cluster-node-tuning-operator/tuned-z9dsn","openshift-dns/node-resolver-rm27j","openshift-image-registry/node-ca-mjjfd","openshift-network-diagnostics/network-check-target-57bm6","openshift-network-operator/iptables-alerter-wczhw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal","openshift-multus/multus-additional-cni-plugins-4pf8v"]
Jul 06 20:23:20.077231 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.077213    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:20.077297 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.077279    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:20.078342 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.078319    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.079688 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.079671    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.080972 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.080951    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.082197 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.081797    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.082516    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pbdcq\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.082759    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.082795    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.082991    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083034    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083059    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083099    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083223    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 06 20:23:20.083470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083375    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l88j2\""
Jul 06 20:23:20.083911 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.083725    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.084583 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.084561    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 06 20:23:20.085112 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085089    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.085200 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085171    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 06 20:23:20.085502 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085477    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 06 20:23:20.085502 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085500    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 06 20:23:20.085698 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085569    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gfv9b\""
Jul 06 20:23:20.085930 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085913    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.086048 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.085999    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.086479 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.086459    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 06 20:23:20.087404 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.087380    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.087821 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.087799    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 06 20:23:20.088485 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.088464    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.088568 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.088484    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5cjxm\""
Jul 06 20:23:20.088568 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.088505    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-r85lv\""
Jul 06 20:23:20.088712 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.088629    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.089143 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089128    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.089231 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089203    2554 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 06 20:23:20.089293 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089271    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 06 20:23:20.089781 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089762    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.089867 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089768    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.089919 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.089872    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dvklp\""
Jul 06 20:23:20.090682 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.090666    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.090773 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.090730    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:20.091424 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.091404    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.091519 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.091495    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.091633 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.091522    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zwd7n\""
Jul 06 20:23:20.092245 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.092224    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.093649 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.093634    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.094018 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.093999    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-ovn\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.094083 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094031    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1004a321-ebe7-4838-88bf-6c0b45512844-tmp-dir\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.094083 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094058    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-kubernetes\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094082    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-var-lib-kubelet\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094098    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f616263c-0395-4994-95de-ec5860eac4a7-host\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.094169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094114    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-conf\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094130    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-tuned\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094152    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-tmp\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094185    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-host-slash\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094210    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094229    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-modprobe-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-systemd-units\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094308    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094308    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1004a321-ebe7-4838-88bf-6c0b45512844-hosts-file\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.094357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094332    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mnw8b\""
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094364    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-systemd\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094384    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-system-cni-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094408    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-bin\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094445    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/effb26aa-de4d-48c5-a38a-67a02809854b-agent-certs\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094494    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-lib-modules\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094549    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-os-release\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094583    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-script-lib\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094609    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x59g6\" (UniqueName: \"kubernetes.io/projected/fc694fa6-b6c6-42bb-a39c-9b8e97857434-kube-api-access-x59g6\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.094632 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094628    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-kubelet\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094645    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-env-overrides\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094660    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36417a94-cd86-478c-9b05-eb89b8746d19-ovn-node-metrics-cert\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094683    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-run\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094726    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-systemd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094756    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094776    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094806    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxg2j\" (UniqueName: \"kubernetes.io/projected/4799b32f-18c9-450e-8724-7cba6bc1bdf1-kube-api-access-vxg2j\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094851    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-etc-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094909    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094948    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hvw5\" (UniqueName: \"kubernetes.io/projected/36417a94-cd86-478c-9b05-eb89b8746d19-kube-api-access-6hvw5\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094971    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khzgq\" (UniqueName: \"kubernetes.io/projected/8dce919a-95e0-4f51-bd85-053330aa2d72-kube-api-access-khzgq\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.094988    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-slash\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095002    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-log-socket\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095024    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095017    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysconfig\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095045    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-sys\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095060    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095076    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-config\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.095071    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095106    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095120    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095137    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095160    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgrnh\" (UniqueName: \"kubernetes.io/projected/f616263c-0395-4994-95de-ec5860eac4a7-kube-api-access-bgrnh\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095180    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-netd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095204    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-host\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095231    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cnibin\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095247    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdrc\" (UniqueName: \"kubernetes.io/projected/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-kube-api-access-2vdrc\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095263    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-node-log\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.095521 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095277    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/effb26aa-de4d-48c5-a38a-67a02809854b-konnectivity-ca\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095294    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pfjn6\" (UniqueName: \"kubernetes.io/projected/1004a321-ebe7-4838-88bf-6c0b45512844-kube-api-access-pfjn6\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095308    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-netns\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095361    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-var-lib-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095385    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-iptables-alerter-script\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095417    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095453    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f616263c-0395-4994-95de-ec5860eac4a7-serviceca\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095554    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095763    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095780    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 06 20:23:20.096300 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.095864    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-hhwzt\""
Jul 06 20:23:20.104097 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.104077    2554 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 06 20:23:20.125306 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.125240    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-05 20:18:19 +0000 UTC" deadline="2028-03-25 06:41:52.985175964 +0000 UTC"
Jul 06 20:23:20.125306 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.125305    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15058h18m32.859875309s"
Jul 06 20:23:20.180783 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.180760    2554 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:20.191310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.191288    2554 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 06 20:23:20.196127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196106    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-kubelet\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196135    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-env-overrides\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196154    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36417a94-cd86-478c-9b05-eb89b8746d19-ovn-node-metrics-cert\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196170    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-run\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.196195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196186    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-systemd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196201    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196218    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196221    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-kubelet\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196243    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxg2j\" (UniqueName: \"kubernetes.io/projected/4799b32f-18c9-450e-8724-7cba6bc1bdf1-kube-api-access-vxg2j\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196270    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-run\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196294    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-etc-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196300    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-systemd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196326    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196355    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hvw5\" (UniqueName: \"kubernetes.io/projected/36417a94-cd86-478c-9b05-eb89b8746d19-kube-api-access-6hvw5\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196363    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196368    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-etc-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.196390 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196384    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khzgq\" (UniqueName: \"kubernetes.io/projected/8dce919a-95e0-4f51-bd85-053330aa2d72-kube-api-access-khzgq\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196419    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196545    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-system-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196578    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-etc-kubernetes\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196607    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-slash\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196632    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-log-socket\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196630    2554 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196682    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysconfig\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196685    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-slash\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196706    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-env-overrides\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196712    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-log-socket\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196719    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-sys\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196744    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysconfig\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196747    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-multus\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196774    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-socket-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196776    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-sys\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196811    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196839    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-config\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.197118 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196869    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cni-binary-copy\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196873    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196894    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-device-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196919    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196938    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196956    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196981    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.196978    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197028    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197054    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-k8s-cni-cncf-io\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197077    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-bin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197095    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197103    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ct977\" (UniqueName: \"kubernetes.io/projected/628372e6-02cc-4e24-a836-90bbc3aa4f6e-kube-api-access-ct977\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197130    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgrnh\" (UniqueName: \"kubernetes.io/projected/f616263c-0395-4994-95de-ec5860eac4a7-kube-api-access-bgrnh\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197176    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-netd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197195    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-host\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197209    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-netns\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.198082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197252    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197271    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-netd\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197281    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cnibin\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197309    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdrc\" (UniqueName: \"kubernetes.io/projected/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-kube-api-access-2vdrc\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197335    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-node-log\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197357    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-config\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197337    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-host\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197362    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-conf-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197393    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-node-log\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197409    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cnibin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197421    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cnibin\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197429    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-os-release\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197445    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-kubelet\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197385    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-binary-copy\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197475    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-dbus\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197486    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4799b32f-18c9-450e-8724-7cba6bc1bdf1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197508    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/effb26aa-de4d-48c5-a38a-67a02809854b-konnectivity-ca\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.199019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197535    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pfjn6\" (UniqueName: \"kubernetes.io/projected/1004a321-ebe7-4838-88bf-6c0b45512844-kube-api-access-pfjn6\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197563    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-netns\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197610    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-var-lib-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197636    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-iptables-alerter-script\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197642    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-netns\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197663    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197675    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-var-lib-openvswitch\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197689    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197715    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f616263c-0395-4994-95de-ec5860eac4a7-serviceca\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.197760    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197769    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-ovn\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197807    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1004a321-ebe7-4838-88bf-6c0b45512844-tmp-dir\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.197834    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:20.697812777 +0000 UTC m=+1.981863315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197867    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-kubernetes\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197876    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-run-ovn\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197893    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-var-lib-kubelet\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197920    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-sys-fs\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197945    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f616263c-0395-4994-95de-ec5860eac4a7-host\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197970    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-conf\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.197993    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-tuned\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198012    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-tmp\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198034    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-socket-dir-parent\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198060    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-daemon-config\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198069    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f616263c-0395-4994-95de-ec5860eac4a7-host\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198083    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-host-slash\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198086    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/effb26aa-de4d-48c5-a38a-67a02809854b-konnectivity-ca\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198110    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198123    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-kubernetes\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198178    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-iptables-alerter-script\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198189    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1004a321-ebe7-4838-88bf-6c0b45512844-tmp-dir\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198195    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f616263c-0395-4994-95de-ec5860eac4a7-serviceca\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198191    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-var-lib-kubelet\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198241    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-run-ovn-kubernetes\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198270    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-modprobe-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198301    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198320    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-sysctl-conf\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198329    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-kubelet-config\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198359    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-systemd-units\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198371    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-modprobe-d\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198417    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1004a321-ebe7-4838-88bf-6c0b45512844-hosts-file\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198426    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-host-slash\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198459    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-systemd-units\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198482    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/1004a321-ebe7-4838-88bf-6c0b45512844-hosts-file\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198493    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-systemd\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198522    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-multus-certs\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198571    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-system-cni-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198598    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-bin\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198583    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-systemd\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198624    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/effb26aa-de4d-48c5-a38a-67a02809854b-agent-certs\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198651    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-lib-modules\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198657    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-system-cni-dir\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.200796 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198679    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-hostroot\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198707    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2mqc\" (UniqueName: \"kubernetes.io/projected/a2c82027-490b-417f-a00e-9720b6f533e4-kube-api-access-m2mqc\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198737    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-os-release\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198762    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-script-lib\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198771    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/fc694fa6-b6c6-42bb-a39c-9b8e97857434-lib-modules\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198787    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x59g6\" (UniqueName: \"kubernetes.io/projected/fc694fa6-b6c6-42bb-a39c-9b8e97857434-kube-api-access-x59g6\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198825    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-registration-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198872    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4799b32f-18c9-450e-8724-7cba6bc1bdf1-os-release\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.198949    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/36417a94-cd86-478c-9b05-eb89b8746d19-host-cni-bin\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.199900    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/36417a94-cd86-478c-9b05-eb89b8746d19-ovnkube-script-lib\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.200642    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/36417a94-cd86-478c-9b05-eb89b8746d19-ovn-node-metrics-cert\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.200834    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-tmp\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.200871    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/fc694fa6-b6c6-42bb-a39c-9b8e97857434-etc-tuned\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.201301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.200984    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/effb26aa-de4d-48c5-a38a-67a02809854b-agent-certs\") pod \"konnectivity-agent-x57bd\" (UID: \"effb26aa-de4d-48c5-a38a-67a02809854b\") " pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.205553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.205514    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-khzgq\" (UniqueName: \"kubernetes.io/projected/8dce919a-95e0-4f51-bd85-053330aa2d72-kube-api-access-khzgq\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.205711 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.205694    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hvw5\" (UniqueName: \"kubernetes.io/projected/36417a94-cd86-478c-9b05-eb89b8746d19-kube-api-access-6hvw5\") pod \"ovnkube-node-59hxf\" (UID: \"36417a94-cd86-478c-9b05-eb89b8746d19\") " pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.205711 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.205699    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxg2j\" (UniqueName: \"kubernetes.io/projected/4799b32f-18c9-450e-8724-7cba6bc1bdf1-kube-api-access-vxg2j\") pod \"multus-additional-cni-plugins-4pf8v\" (UID: \"4799b32f-18c9-450e-8724-7cba6bc1bdf1\") " pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.208395 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.208372    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdrc\" (UniqueName: \"kubernetes.io/projected/4b5bbe0c-e896-407c-b7ef-7267dc11d1c8-kube-api-access-2vdrc\") pod \"iptables-alerter-wczhw\" (UID: \"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8\") " pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.210067 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.210051    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:20.210171 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.210075    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:20.210171 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.210088    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:20.210171 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.210156    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:20.710135763 +0000 UTC m=+1.994186304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:20.212365 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.212329    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pfjn6\" (UniqueName: \"kubernetes.io/projected/1004a321-ebe7-4838-88bf-6c0b45512844-kube-api-access-pfjn6\") pod \"node-resolver-rm27j\" (UID: \"1004a321-ebe7-4838-88bf-6c0b45512844\") " pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.212448 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.212383    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgrnh\" (UniqueName: \"kubernetes.io/projected/f616263c-0395-4994-95de-ec5860eac4a7-kube-api-access-bgrnh\") pod \"node-ca-mjjfd\" (UID: \"f616263c-0395-4994-95de-ec5860eac4a7\") " pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.212686 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.212669    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x59g6\" (UniqueName: \"kubernetes.io/projected/fc694fa6-b6c6-42bb-a39c-9b8e97857434-kube-api-access-x59g6\") pod \"tuned-z9dsn\" (UID: \"fc694fa6-b6c6-42bb-a39c-9b8e97857434\") " pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.225315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.225296    2554 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pldgr"
Jul 06 20:23:20.231256 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.231235    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-z9dsn"
Jul 06 20:23:20.238715 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.238697    2554 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pldgr"
Jul 06 20:23:20.300134 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300103    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-registration-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300142    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-system-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300172    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-etc-kubernetes\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300221    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-etc-kubernetes\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300235    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-system-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300232    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-registration-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300294    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-multus\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300331    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-socket-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300349    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cni-binary-copy\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300366    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-device-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300408    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300413    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-multus\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300434    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-k8s-cni-cncf-io\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300441    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-device-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.300444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300457    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-bin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300489    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-k8s-cni-cncf-io\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300486    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ct977\" (UniqueName: \"kubernetes.io/projected/628372e6-02cc-4e24-a836-90bbc3aa4f6e-kube-api-access-ct977\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300492    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-cni-bin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300554    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-netns\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300574    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300578    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-socket-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300604    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-conf-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300626    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-cni-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300629    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cnibin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300663    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-os-release\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300672    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cnibin\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300672    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-conf-dir\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300631    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-netns\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300724    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-kubelet\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300689    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-var-lib-kubelet\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.300693    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300757    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-dbus\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.301205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300783    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-os-release\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.300811    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:20.800790482 +0000 UTC m=+2.084841015 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300833    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300848    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-dbus\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300897    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300901    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-sys-fs\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300935    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-sys-fs\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300940    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-socket-dir-parent\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300969    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-daemon-config\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300976    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-socket-dir-parent\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300983    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-cni-binary-copy\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.300999    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301025    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-kubelet-config\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301054    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-multus-certs\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301079    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-hostroot\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301099    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/59ae90f0-b70b-4374-90ac-d84276e8bccd-kubelet-config\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.301816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301101    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2mqc\" (UniqueName: \"kubernetes.io/projected/a2c82027-490b-417f-a00e-9720b6f533e4-kube-api-access-m2mqc\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.302310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301131    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-host-run-multus-certs\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.302310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301144    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a2c82027-490b-417f-a00e-9720b6f533e4-etc-selinux\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.302310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301161    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/628372e6-02cc-4e24-a836-90bbc3aa4f6e-hostroot\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.302310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.301306    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/628372e6-02cc-4e24-a836-90bbc3aa4f6e-multus-daemon-config\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.311340 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.311309    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2mqc\" (UniqueName: \"kubernetes.io/projected/a2c82027-490b-417f-a00e-9720b6f533e4-kube-api-access-m2mqc\") pod \"aws-ebs-csi-driver-node-ghh5t\" (UID: \"a2c82027-490b-417f-a00e-9720b6f533e4\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.311442 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.311367    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ct977\" (UniqueName: \"kubernetes.io/projected/628372e6-02cc-4e24-a836-90bbc3aa4f6e-kube-api-access-ct977\") pod \"multus-zb467\" (UID: \"628372e6-02cc-4e24-a836-90bbc3aa4f6e\") " pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.396578 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.396520    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e41b390cac1ae2594606d5b623e99b.slice/crio-c147be6476f034afd87de4894ae03edd7d2c3263dfcf8739e1ee50dd2e43d440 WatchSource:0}: Error finding container c147be6476f034afd87de4894ae03edd7d2c3263dfcf8739e1ee50dd2e43d440: Status 404 returned error can't find the container with id c147be6476f034afd87de4894ae03edd7d2c3263dfcf8739e1ee50dd2e43d440
Jul 06 20:23:20.397577 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.397550    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ef1a50927e5ff318f1211f698ffaa1.slice/crio-ede5684ba06a8c79385f597dafd3125602d1886f8b6e5a8a8fc41bb6c4def687 WatchSource:0}: Error finding container ede5684ba06a8c79385f597dafd3125602d1886f8b6e5a8a8fc41bb6c4def687: Status 404 returned error can't find the container with id ede5684ba06a8c79385f597dafd3125602d1886f8b6e5a8a8fc41bb6c4def687
Jul 06 20:23:20.402856 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.402841    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:23:20.411847 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.411829    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wczhw"
Jul 06 20:23:20.420009 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.419979    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b5bbe0c_e896_407c_b7ef_7267dc11d1c8.slice/crio-c86b630082ff42fb166f6178f0bdc22d1af774abe01e2ff5abbf7cea42fc271d WatchSource:0}: Error finding container c86b630082ff42fb166f6178f0bdc22d1af774abe01e2ff5abbf7cea42fc271d: Status 404 returned error can't find the container with id c86b630082ff42fb166f6178f0bdc22d1af774abe01e2ff5abbf7cea42fc271d
Jul 06 20:23:20.441524 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.441503    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:20.449730 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.449702    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36417a94_cd86_478c_9b05_eb89b8746d19.slice/crio-5d50cbe510efc9d83cf62685b2dfec0910a1b8717ea3993a637cf70c2f400c1c WatchSource:0}: Error finding container 5d50cbe510efc9d83cf62685b2dfec0910a1b8717ea3993a637cf70c2f400c1c: Status 404 returned error can't find the container with id 5d50cbe510efc9d83cf62685b2dfec0910a1b8717ea3993a637cf70c2f400c1c
Jul 06 20:23:20.457128 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.457109    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-4pf8v"
Jul 06 20:23:20.461425 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.461408    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:20.465248 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.465228    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4799b32f_18c9_450e_8724_7cba6bc1bdf1.slice/crio-185c1b261921067646605b5b0e7f1daaef18c617245044521c896045ef4d3094 WatchSource:0}: Error finding container 185c1b261921067646605b5b0e7f1daaef18c617245044521c896045ef4d3094: Status 404 returned error can't find the container with id 185c1b261921067646605b5b0e7f1daaef18c617245044521c896045ef4d3094
Jul 06 20:23:20.468955 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.468932    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeffb26aa_de4d_48c5_a38a_67a02809854b.slice/crio-eb3019234eb93a9565c78c9347567d394f379b7382ad592b4f9a077b40662d2c WatchSource:0}: Error finding container eb3019234eb93a9565c78c9347567d394f379b7382ad592b4f9a077b40662d2c: Status 404 returned error can't find the container with id eb3019234eb93a9565c78c9347567d394f379b7382ad592b4f9a077b40662d2c
Jul 06 20:23:20.487580 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.487559    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rm27j"
Jul 06 20:23:20.492262 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.492244    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mjjfd"
Jul 06 20:23:20.496636 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.496611    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1004a321_ebe7_4838_88bf_6c0b45512844.slice/crio-70a705c44cb8ed9e624a2e14eb26927e8dc78cdf71cd3f95998bc749a859c757 WatchSource:0}: Error finding container 70a705c44cb8ed9e624a2e14eb26927e8dc78cdf71cd3f95998bc749a859c757: Status 404 returned error can't find the container with id 70a705c44cb8ed9e624a2e14eb26927e8dc78cdf71cd3f95998bc749a859c757
Jul 06 20:23:20.501254 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.501236    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf616263c_0395_4994_95de_ec5860eac4a7.slice/crio-d967b18f3911e13b2be106392f72f267165a25e4139a6876df90c304c2325b65 WatchSource:0}: Error finding container d967b18f3911e13b2be106392f72f267165a25e4139a6876df90c304c2325b65: Status 404 returned error can't find the container with id d967b18f3911e13b2be106392f72f267165a25e4139a6876df90c304c2325b65
Jul 06 20:23:20.543862 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.543820    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zb467"
Jul 06 20:23:20.549671 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.549645    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t"
Jul 06 20:23:20.552044 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.552021    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod628372e6_02cc_4e24_a836_90bbc3aa4f6e.slice/crio-e42aa533e5485c2ef01ec388f84b31535cc3b7edd4d01817696df7764c5f33d6 WatchSource:0}: Error finding container e42aa533e5485c2ef01ec388f84b31535cc3b7edd4d01817696df7764c5f33d6: Status 404 returned error can't find the container with id e42aa533e5485c2ef01ec388f84b31535cc3b7edd4d01817696df7764c5f33d6
Jul 06 20:23:20.557048 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.557027    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2c82027_490b_417f_a00e_9720b6f533e4.slice/crio-d56cdfd4949e8e754c52f69f1f96885448b4e6bcde77d465560d126b0adc0d5f WatchSource:0}: Error finding container d56cdfd4949e8e754c52f69f1f96885448b4e6bcde77d465560d126b0adc0d5f: Status 404 returned error can't find the container with id d56cdfd4949e8e754c52f69f1f96885448b4e6bcde77d465560d126b0adc0d5f
Jul 06 20:23:20.676907 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:20.676878    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc694fa6_b6c6_42bb_a39c_9b8e97857434.slice/crio-fd2dcb16d3e9882479dc0d8efe7cd3686ace0b34d8aa336512d9e70c570cde02 WatchSource:0}: Error finding container fd2dcb16d3e9882479dc0d8efe7cd3686ace0b34d8aa336512d9e70c570cde02: Status 404 returned error can't find the container with id fd2dcb16d3e9882479dc0d8efe7cd3686ace0b34d8aa336512d9e70c570cde02
Jul 06 20:23:20.703734 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.703707    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:20.703849 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.703835    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:20.703932 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.703896    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:21.703875175 +0000 UTC m=+2.987925697 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:20.804799 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.804764    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:20.804799 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:20.804805    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:20.805054 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.804955    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:20.805054 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.804984    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:20.805054 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.805008    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:20.805054 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.805017    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:21.8049996 +0000 UTC m=+3.089050138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:20.805054 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.805024    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:20.805256 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:20.805083    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:21.805063927 +0000 UTC m=+3.089114466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:21.052516 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.052436    2554 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:21.240860 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.240769    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:18:20 +0000 UTC" deadline="2028-01-18 13:04:04.868309986 +0000 UTC"
Jul 06 20:23:21.240860 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.240801    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13456h40m43.627514061s"
Jul 06 20:23:21.278866 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.278155    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:21.278866 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.278303    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:21.278866 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.278682    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:21.278866 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.278798    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:21.293493 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.293402    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9dsn" event={"ID":"fc694fa6-b6c6-42bb-a39c-9b8e97857434","Type":"ContainerStarted","Data":"fd2dcb16d3e9882479dc0d8efe7cd3686ace0b34d8aa336512d9e70c570cde02"}
Jul 06 20:23:21.308047 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.307957    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t" event={"ID":"a2c82027-490b-417f-a00e-9720b6f533e4","Type":"ContainerStarted","Data":"d56cdfd4949e8e754c52f69f1f96885448b4e6bcde77d465560d126b0adc0d5f"}
Jul 06 20:23:21.330016 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.329901    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zb467" event={"ID":"628372e6-02cc-4e24-a836-90bbc3aa4f6e","Type":"ContainerStarted","Data":"e42aa533e5485c2ef01ec388f84b31535cc3b7edd4d01817696df7764c5f33d6"}
Jul 06 20:23:21.338103 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.338068    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rm27j" event={"ID":"1004a321-ebe7-4838-88bf-6c0b45512844","Type":"ContainerStarted","Data":"70a705c44cb8ed9e624a2e14eb26927e8dc78cdf71cd3f95998bc749a859c757"}
Jul 06 20:23:21.341129 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.341098    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerStarted","Data":"185c1b261921067646605b5b0e7f1daaef18c617245044521c896045ef4d3094"}
Jul 06 20:23:21.357917 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.357886    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"5d50cbe510efc9d83cf62685b2dfec0910a1b8717ea3993a637cf70c2f400c1c"}
Jul 06 20:23:21.365112 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.365082    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"51e41b390cac1ae2594606d5b623e99b","Type":"ContainerStarted","Data":"c147be6476f034afd87de4894ae03edd7d2c3263dfcf8739e1ee50dd2e43d440"}
Jul 06 20:23:21.368873 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.368687    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mjjfd" event={"ID":"f616263c-0395-4994-95de-ec5860eac4a7","Type":"ContainerStarted","Data":"d967b18f3911e13b2be106392f72f267165a25e4139a6876df90c304c2325b65"}
Jul 06 20:23:21.375186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.373060    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x57bd" event={"ID":"effb26aa-de4d-48c5-a38a-67a02809854b","Type":"ContainerStarted","Data":"eb3019234eb93a9565c78c9347567d394f379b7382ad592b4f9a077b40662d2c"}
Jul 06 20:23:21.386257 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.386222    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wczhw" event={"ID":"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8","Type":"ContainerStarted","Data":"c86b630082ff42fb166f6178f0bdc22d1af774abe01e2ff5abbf7cea42fc271d"}
Jul 06 20:23:21.391594 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.391566    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" event={"ID":"61ef1a50927e5ff318f1211f698ffaa1","Type":"ContainerStarted","Data":"ede5684ba06a8c79385f597dafd3125602d1886f8b6e5a8a8fc41bb6c4def687"}
Jul 06 20:23:21.711682 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.711591    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:21.711867 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.711802    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:21.711933 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.711874    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:23.711851713 +0000 UTC m=+4.995902237 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:21.812040 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.812004    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:21.812230 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:21.812059    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:21.812230 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812222    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:21.812347 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812288    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:23.812267623 +0000 UTC m=+5.096318146 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:21.812841 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812806    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:21.812841 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812834    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:21.812841 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812848    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:21.813062 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:21.812896    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:23.812880327 +0000 UTC m=+5.096930867 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:22.241423 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:22.241376    2554 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-05 20:18:20 +0000 UTC" deadline="2028-01-07 03:49:10.736432552 +0000 UTC"
Jul 06 20:23:22.241423 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:22.241420    2554 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13183h25m48.495017509s"
Jul 06 20:23:22.275426 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:22.275394    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:22.275625 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:22.275571    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:22.504784 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:22.504698    2554 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 06 20:23:23.278872 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:23.278145    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:23.278872 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.278291    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:23.278872 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:23.278719    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:23.278872 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.278812    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:23.728169 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:23.727511    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:23.728169 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.727697    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:23.728169 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.727766    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:27.727747137 +0000 UTC m=+9.011797671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:23.828142    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:23.828199    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828360    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828432    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:27.828406623 +0000 UTC m=+9.112457143 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828847    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828871    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828885    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:23.829020 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:23.828931    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:27.828914315 +0000 UTC m=+9.112964833 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:24.275971 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:24.275887    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:24.276140 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:24.276060    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:25.276487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:25.276443    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:25.277163 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:25.276594    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:25.277163 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:25.277101    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:25.277353 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:25.277193    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:26.276852 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:26.276296    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:26.276852 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:26.276440    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:27.276005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:27.275975    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:27.276190 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.276108    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:27.276573 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:27.276553    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:27.276675 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.276654    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:27.757857 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:27.757818    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:27.758266 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.757957    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:27.758266 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.758023    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:35.758006783 +0000 UTC m=+17.042057325 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:27.859140 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:27.859107    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:27.859312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:27.859163    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:27.859312 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859306    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:27.859312 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859312    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:27.859312 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859343    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:27.859580 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859357    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:27.859580 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859368    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:35.859349892 +0000 UTC m=+17.143400431 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:27.859580 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:27.859409    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:35.859392285 +0000 UTC m=+17.143442817 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:28.275432 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:28.275299    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:28.275635 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:28.275442    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:29.276768 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:29.276707    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:29.277202 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:29.276837    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:29.277202 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:29.276880    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:29.277202 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:29.276986    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:30.275990 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:30.275955    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:30.276196 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:30.276107    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:31.275659 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:31.275626    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:31.276115 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:31.275629    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:31.276115 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:31.275786    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:31.276115 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:31.275851    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:32.276210 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:32.275960    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:32.276728 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:32.276315    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:32.427406 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:32.427319    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" event={"ID":"61ef1a50927e5ff318f1211f698ffaa1","Type":"ContainerStarted","Data":"260997656aee868b00652ffecc75b0028a9e5a6ea08a95d8afd6bcc2725dc817"}
Jul 06 20:23:32.429334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:32.429302    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-z9dsn" event={"ID":"fc694fa6-b6c6-42bb-a39c-9b8e97857434","Type":"ContainerStarted","Data":"f57538778fb9ebc6cbe24b8bf39b348d110b7b69fd2f13032d4ea9620b3f84e7"}
Jul 06 20:23:32.442738 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:32.442689    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-47.ec2.internal" podStartSLOduration=12.442668475 podStartE2EDuration="12.442668475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:32.442246913 +0000 UTC m=+13.726297455" watchObservedRunningTime="2026-07-06 20:23:32.442668475 +0000 UTC m=+13.726719019"
Jul 06 20:23:33.276420 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.276380    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:33.276870 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.276393    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:33.276870 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:33.276523    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:33.276870 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:33.276624    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:33.434797 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.434293    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rm27j" event={"ID":"1004a321-ebe7-4838-88bf-6c0b45512844","Type":"ContainerStarted","Data":"5319b7cb89b1a7126937d7c49642ebd23748d139b62a6007b919976d47ab1dc3"}
Jul 06 20:23:33.436186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.436160    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="4cdda2fc6a47193d95358e0058265d0cc09d8c1c076bd923e952a3cf38863a93" exitCode=0
Jul 06 20:23:33.436281 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.436244    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"4cdda2fc6a47193d95358e0058265d0cc09d8c1c076bd923e952a3cf38863a93"}
Jul 06 20:23:33.438283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.438261    2554 generic.go:356] "Generic (PLEG): container finished" podID="51e41b390cac1ae2594606d5b623e99b" containerID="f2bf6159856e2b4c1f426045104bcc1bfb4efc2459858c6d9569158495a58663" exitCode=0
Jul 06 20:23:33.438411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.438341    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"51e41b390cac1ae2594606d5b623e99b","Type":"ContainerDied","Data":"f2bf6159856e2b4c1f426045104bcc1bfb4efc2459858c6d9569158495a58663"}
Jul 06 20:23:33.440072 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.440022    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mjjfd" event={"ID":"f616263c-0395-4994-95de-ec5860eac4a7","Type":"ContainerStarted","Data":"204834a1d1b09499fc10274ec25712b52c03b3189046e699346383590efada4b"}
Jul 06 20:23:33.441572 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.441549    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-x57bd" event={"ID":"effb26aa-de4d-48c5-a38a-67a02809854b","Type":"ContainerStarted","Data":"5e4f179aae1ffb3bea72822b18d452471feca6f07fab2a125840d952f995db27"}
Jul 06 20:23:33.443433 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.443409    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wczhw" event={"ID":"4b5bbe0c-e896-407c-b7ef-7267dc11d1c8","Type":"ContainerStarted","Data":"4d37112779cc90d5bd9f8d7fe1d911a5eddcadda8ab16ce234fe27d47376276f"}
Jul 06 20:23:33.445332 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.445310    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t" event={"ID":"a2c82027-490b-417f-a00e-9720b6f533e4","Type":"ContainerStarted","Data":"e43abf321c683318f1dbe97c4609db322bc4d247508e20cc95892775f0f0d6bd"}
Jul 06 20:23:33.449285 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.449243    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-z9dsn" podStartSLOduration=3.522724874 podStartE2EDuration="14.449231778s" totalImagesPullingTime="10.926506904s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:32.458830118 +0000 UTC m=+13.742880654" watchObservedRunningTime="2026-07-06 20:23:33.449231778 +0000 UTC m=+14.733282367"
Jul 06 20:23:33.449380 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.449343    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rm27j" podStartSLOduration=3.378608647 podStartE2EDuration="14.449339799s" totalImagesPullingTime="11.070731152s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:33.448929005 +0000 UTC m=+14.732979546" watchObservedRunningTime="2026-07-06 20:23:33.449339799 +0000 UTC m=+14.733390340"
Jul 06 20:23:33.504548 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.504480    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wczhw" podStartSLOduration=3.355199514 podStartE2EDuration="14.504458495s" totalImagesPullingTime="11.149258981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:33.504348451 +0000 UTC m=+14.788398987" watchObservedRunningTime="2026-07-06 20:23:33.504458495 +0000 UTC m=+14.788509036"
Jul 06 20:23:33.521491 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.521309    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-x57bd" podStartSLOduration=3.433630234 podStartE2EDuration="14.521295608s" totalImagesPullingTime="11.087665374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:33.52119267 +0000 UTC m=+14.805243205" watchObservedRunningTime="2026-07-06 20:23:33.521295608 +0000 UTC m=+14.805346148"
Jul 06 20:23:33.537023 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:33.536978    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mjjfd" podStartSLOduration=3.470339056 podStartE2EDuration="14.53695598s" totalImagesPullingTime="11.066616924s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:33.536726143 +0000 UTC m=+14.820776685" watchObservedRunningTime="2026-07-06 20:23:33.53695598 +0000 UTC m=+14.821006521"
Jul 06 20:23:34.275704 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:34.275669    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:34.275881 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:34.275800    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:35.275592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:35.275552    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:35.275592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:35.275589    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:35.276080 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.275704    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:35.276080 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.275843    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:35.820610 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:35.820577    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:35.820826 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.820712    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:35.820826 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.820767    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:51.820752275 +0000 UTC m=+33.104802798 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:35.921218 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:35.921184    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:35.921405 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:35.921287    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:35.921405 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921331    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:35.921495 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921407    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:23:51.921386173 +0000 UTC m=+33.205436696 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:35.921495 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921411    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:35.921495 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921442    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:35.921495 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921455    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:35.921495 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:35.921492    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:23:51.921480102 +0000 UTC m=+33.205530632 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:36.276159 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:36.276085    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:36.276649 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:36.276216    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:37.276092 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:37.276052    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:37.276348 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:37.276200    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:37.276348 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:37.276195    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:37.276348 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:37.276310    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:38.173236 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:38.173195    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:38.174073 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:38.174045    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:38.275805 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:38.275761    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:38.275996 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:38.275910    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:38.455644 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:38.455563    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:38.456198 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:38.456172    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-x57bd"
Jul 06 20:23:39.278859 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:39.278827    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:39.279038 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:39.278944    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:39.279038 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:39.278992    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:39.279165 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:39.279104    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:40.275401 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:40.275336    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:40.275885 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:40.275478    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:41.279315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:41.279260    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:41.279788 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:41.279392    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:41.279788 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:41.279260    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:41.279788 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:41.279579    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:42.276078 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.275900    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:42.276215 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:42.276191    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:42.465167 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.465134    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zb467" event={"ID":"628372e6-02cc-4e24-a836-90bbc3aa4f6e","Type":"ContainerStarted","Data":"51625c7173beba7834d7b569a9ee800d52757e5d103da941a2ebf1fa7f930598"}
Jul 06 20:23:42.466961 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.466940    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:23:42.467260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.467239    2554 generic.go:356] "Generic (PLEG): container finished" podID="36417a94-cd86-478c-9b05-eb89b8746d19" containerID="cb62eaa10b8c973af147d40790f204b8822e810ac328dbb79b56098b8df3d2af" exitCode=1
Jul 06 20:23:42.467332 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.467310    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"80681af78adb14a709bcdfb164f4d0b2304728e1d623e8515301bf7ea2460d08"}
Jul 06 20:23:42.467407 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.467394    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerDied","Data":"cb62eaa10b8c973af147d40790f204b8822e810ac328dbb79b56098b8df3d2af"}
Jul 06 20:23:42.467472 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.467449    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"ddfb20a6adaebac68dc67335e2bff3d278f75ffbf9173bc4d5b6a7580e34e880"}
Jul 06 20:23:42.468817 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.468797    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" event={"ID":"51e41b390cac1ae2594606d5b623e99b","Type":"ContainerStarted","Data":"7db508bb3b91de031025851981821199270975f82177953dd139daca87fdc3cc"}
Jul 06 20:23:42.482502 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.482466    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zb467" podStartSLOduration=2.170063589 podStartE2EDuration="23.482455895s" totalImagesPullingTime="21.312392306s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:42.482175213 +0000 UTC m=+23.766225764" watchObservedRunningTime="2026-07-06 20:23:42.482455895 +0000 UTC m=+23.766506436"
Jul 06 20:23:42.499446 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.499406    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-47.ec2.internal" podStartSLOduration=22.499377096 podStartE2EDuration="22.499377096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:42.49918738 +0000 UTC m=+23.783237924" watchObservedRunningTime="2026-07-06 20:23:42.499377096 +0000 UTC m=+23.783427639"
Jul 06 20:23:42.871703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:42.871520    2554 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 06 20:23:43.275784 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.275708    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:43.275932 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:43.275818    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:43.275932 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.275873    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:43.276000 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:43.275954    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:43.305763 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.305646    2554 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-06T20:23:42.871693604Z","UUID":"1623f655-74f8-4bc6-a8b9-7762aaf946cf","Handler":null,"Name":"","Endpoint":""}
Jul 06 20:23:43.307370 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.307339    2554 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 06 20:23:43.307501 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.307378    2554 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 06 20:23:43.473920 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.473892    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:23:43.474363 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.474322    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"2dbb34bc25c91c87b23246312f011d4a752f6adde2cb1037c0aad2586ea9c7a7"}
Jul 06 20:23:43.474420 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.474405    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"9b433450a8e9d98efa9100e0bce61bc01003a09369b93b6ca01767022128d035"}
Jul 06 20:23:43.474470 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.474458    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"16f97b9ce3aa35f16e861671a3d28478cd536245c5c9c1e22e3dc9ba2dcf1572"}
Jul 06 20:23:43.476126 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:43.476102    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t" event={"ID":"a2c82027-490b-417f-a00e-9720b6f533e4","Type":"ContainerStarted","Data":"8cea22a2b603a974f774898050175baef6e0708c66c2bd35570c3655854accdf"}
Jul 06 20:23:44.275592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:44.275560    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:44.275781 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:44.275681    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:45.276466 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:45.276386    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:45.276882 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:45.276518    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:45.276882 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:45.276574    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:45.276882 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:45.276685    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:46.275675 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.275646    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:46.275823 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:46.275771    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:46.483446 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.483410    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t" event={"ID":"a2c82027-490b-417f-a00e-9720b6f533e4","Type":"ContainerStarted","Data":"b654422ad3e473ac78b59fccb23b0b3896c04664d92e733284d267e171b6192c"}
Jul 06 20:23:46.485056 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.485027    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="1180bbe5150aa3e52425d380f10ba97276fdf38f92c2247cdb00dbeddbdf43af" exitCode=0
Jul 06 20:23:46.485183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.485103    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"1180bbe5150aa3e52425d380f10ba97276fdf38f92c2247cdb00dbeddbdf43af"}
Jul 06 20:23:46.488251 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.488233    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:23:46.488601 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.488579    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"92de342b3b3d86cf52fe9026d7f07a0ee465cc9f9835d2f528e9219dd67b5e7b"}
Jul 06 20:23:46.503519 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:46.503483    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ghh5t" podStartSLOduration=2.8350336990000002 podStartE2EDuration="26.503469338s" totalImagesPullingTime="23.668435639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:46.503450298 +0000 UTC m=+27.787500838" watchObservedRunningTime="2026-07-06 20:23:46.503469338 +0000 UTC m=+27.787519871"
Jul 06 20:23:47.275795 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:47.275757    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:47.275795 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:47.275758    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:47.276037 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:47.275880    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:47.276037 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:47.275935    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:48.276282 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.276104    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:48.276916 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:48.276377    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:48.494948 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.494917    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="3c109b8674a1c009f8b115aa1e622f4fb15fb6f7b96fe980815edff367aec13b" exitCode=0
Jul 06 20:23:48.495101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.495000    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"3c109b8674a1c009f8b115aa1e622f4fb15fb6f7b96fe980815edff367aec13b"}
Jul 06 20:23:48.498458 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.498443    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:23:48.498818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.498803    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"8152f61e806471b726a8d8486a807ab9fe06a7dcb8a5189e2d2dff1b72772b0a"}
Jul 06 20:23:48.499183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.499155    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:48.499290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.499251    2554 scope.go:117] "RemoveContainer" containerID="cb62eaa10b8c973af147d40790f204b8822e810ac328dbb79b56098b8df3d2af"
Jul 06 20:23:48.514747 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:48.514729    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:49.276770 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.276737    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:49.277203 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:49.276867    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:49.277203 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.276912    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:49.277203 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:49.277021    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:49.504331 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.504303    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:23:49.504707 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.504683    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" event={"ID":"36417a94-cd86-478c-9b05-eb89b8746d19","Type":"ContainerStarted","Data":"d8386c7c410434600ee79ed1d1dd1d65c345c24eb257c8a82b35a4e4413f6cde"}
Jul 06 20:23:49.504901 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.504880    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:49.505023 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.504939    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:49.519819 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.519794    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:23:49.532286 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:49.532212    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf" podStartSLOduration=9.082616816 podStartE2EDuration="30.53219668s" totalImagesPullingTime="21.449579864s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:23:49.530743184 +0000 UTC m=+30.814793732" watchObservedRunningTime="2026-07-06 20:23:49.53219668 +0000 UTC m=+30.816247221"
Jul 06 20:23:50.276007 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.275980    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:50.276178 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:50.276153    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:50.437831 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.437796    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57bm6"]
Jul 06 20:23:50.438220 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.437941    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:50.438220 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:50.438052    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:50.440569 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.440521    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k74c5"]
Jul 06 20:23:50.451699 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.451675    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xr6rj"]
Jul 06 20:23:50.451808 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.451767    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:50.451849 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:50.451836    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:50.509410 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.509370    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="454a5d99018dbf242bb9951bec99bd4811cdbb6c5e3443979eb89fc7c1195543" exitCode=0
Jul 06 20:23:50.509574 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.509453    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:50.509574 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:50.509450    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"454a5d99018dbf242bb9951bec99bd4811cdbb6c5e3443979eb89fc7c1195543"}
Jul 06 20:23:50.509703 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:50.509618    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:51.841849 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:51.841818    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:51.842398 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.841977    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:51.842398 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.842061    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:23.842042319 +0000 UTC m=+65.126092842 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 06 20:23:51.942602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:51.942559    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:51.942759 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:51.942618    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:51.942759 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942756    2554 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:51.942874 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942801    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 06 20:23:51.942874 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942820    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret podName:59ae90f0-b70b-4374-90ac-d84276e8bccd nodeName:}" failed. No retries permitted until 2026-07-06 20:24:23.942803031 +0000 UTC m=+65.226853557 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret") pod "global-pull-secret-syncer-xr6rj" (UID: "59ae90f0-b70b-4374-90ac-d84276e8bccd") : object "kube-system"/"original-pull-secret" not registered
Jul 06 20:23:51.942874 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942829    2554 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 06 20:23:51.942874 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942842    2554 projected.go:196] Error preparing data for projected volume kube-api-access-mcv5r for pod openshift-network-diagnostics/network-check-target-57bm6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:51.943013 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:51.942894    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r podName:d2d04775-be12-4888-a7a4-ac81c6e5bfdb nodeName:}" failed. No retries permitted until 2026-07-06 20:24:23.942877978 +0000 UTC m=+65.226928499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mcv5r" (UniqueName: "kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r") pod "network-check-target-57bm6" (UID: "d2d04775-be12-4888-a7a4-ac81c6e5bfdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 06 20:23:52.275606 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:52.275416    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:52.275740 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:52.275439    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:52.275740 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:52.275682    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:52.275801 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:52.275465    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:52.275846 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:52.275736    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:52.275846 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:52.275819    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:54.275717 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.275670    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:54.276234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.275670    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:54.276234 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.275788    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-57bm6" podUID="d2d04775-be12-4888-a7a4-ac81c6e5bfdb"
Jul 06 20:23:54.276234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.275670    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:54.276234 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.275899    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-k74c5" podUID="8dce919a-95e0-4f51-bd85-053330aa2d72"
Jul 06 20:23:54.276234 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.275976    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-xr6rj" podUID="59ae90f0-b70b-4374-90ac-d84276e8bccd"
Jul 06 20:23:54.502380 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.502345    2554 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-47.ec2.internal" event="NodeReady"
Jul 06 20:23:54.502571 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.502500    2554 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 06 20:23:54.537155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.537084    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"]
Jul 06 20:23:54.542804 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.542769    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"]
Jul 06 20:23:54.542982 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.542897    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.544922 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.544900    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"]
Jul 06 20:23:54.545085 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.545045    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.545719 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.545696    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 06 20:23:54.545830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.545732    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 06 20:23:54.545830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.545753    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 06 20:23:54.545944 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.545696    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-whsq2\""
Jul 06 20:23:54.546018 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.546002    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 06 20:23:54.546666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.546646    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"]
Jul 06 20:23:54.546792 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.546775    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.547379 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.547360    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 06 20:23:54.547913 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.547892    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 06 20:23:54.548010 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.547988    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 06 20:23:54.548876 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.548858    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.549849 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.549822    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 06 20:23:54.551525 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.551504    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 06 20:23:54.552078 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.552056    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zmkh5\""
Jul 06 20:23:54.552174 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.552091    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"]
Jul 06 20:23:54.552498 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.552428    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 06 20:23:54.552498 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.552447    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 06 20:23:54.554479 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.554125    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 06 20:23:54.558552 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.558510    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"]
Jul 06 20:23:54.559229 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.559206    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-dsztb"]
Jul 06 20:23:54.559408 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.559387    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 06 20:23:54.561798 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.561750    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"]
Jul 06 20:23:54.562004 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.561872    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.562513 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.562490    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-cfjph"]
Jul 06 20:23:54.564643 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.564621    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 06 20:23:54.564794 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.564654    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 06 20:23:54.565020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.565000    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.565147 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.565038    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 06 20:23:54.565855 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.565729    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gmpj7\""
Jul 06 20:23:54.568371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.568351    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mqhkz\""
Jul 06 20:23:54.568610 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.568592    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 06 20:23:54.568610 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.568607    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 06 20:23:54.576031 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.575946    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"]
Jul 06 20:23:54.577070 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.576688    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dsztb"]
Jul 06 20:23:54.577608 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.577588    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cfjph"]
Jul 06 20:23:54.664649 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664620    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.664649 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664654    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664674    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664751    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664768    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp4zg\" (UniqueName: \"kubernetes.io/projected/78151588-e645-407d-ab18-f0e5870972f6-kube-api-access-gp4zg\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664816    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thbdw\" (UniqueName: \"kubernetes.io/projected/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-kube-api-access-thbdw\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664860    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.664899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664888    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78151588-e645-407d-ab18-f0e5870972f6-cert\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664930    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664960    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrhv7\" (UniqueName: \"kubernetes.io/projected/2722582d-cd19-48d0-8c10-0f26029db59b-kube-api-access-vrhv7\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664975    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.664992    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665017    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665032    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9d2z\" (UniqueName: \"kubernetes.io/projected/e234fbf6-e9ba-410d-94c2-b780b451b92d-kube-api-access-t9d2z\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665055    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8791898b-dc06-4841-8080-26b5d949fdaf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665082    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxxfz\" (UniqueName: \"kubernetes.io/projected/8791898b-dc06-4841-8080-26b5d949fdaf-kube-api-access-rxxfz\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.665150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665150    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z7rkm\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665172    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665201    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665253    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665297    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2722582d-cd19-48d0-8c10-0f26029db59b-tmp-dir\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665315    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2722582d-cd19-48d0-8c10-0f26029db59b-config-volume\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665329    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8791898b-dc06-4841-8080-26b5d949fdaf-tmp\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665345    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e234fbf6-e9ba-410d-94c2-b780b451b92d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.665477 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.665372    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766044 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766000    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766049    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp4zg\" (UniqueName: \"kubernetes.io/projected/78151588-e645-407d-ab18-f0e5870972f6-kube-api-access-gp4zg\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766077    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thbdw\" (UniqueName: \"kubernetes.io/projected/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-kube-api-access-thbdw\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766108    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766131    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78151588-e645-407d-ab18-f0e5870972f6-cert\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766158    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766185    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrhv7\" (UniqueName: \"kubernetes.io/projected/2722582d-cd19-48d0-8c10-0f26029db59b-kube-api-access-vrhv7\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766209    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.766234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766236    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766273    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766299    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9d2z\" (UniqueName: \"kubernetes.io/projected/e234fbf6-e9ba-410d-94c2-b780b451b92d-kube-api-access-t9d2z\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766331    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8791898b-dc06-4841-8080-26b5d949fdaf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766354    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxxfz\" (UniqueName: \"kubernetes.io/projected/8791898b-dc06-4841-8080-26b5d949fdaf-kube-api-access-rxxfz\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766386    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z7rkm\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766411    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766440    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766469    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.766484    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766501    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2722582d-cd19-48d0-8c10-0f26029db59b-tmp-dir\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766554    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2722582d-cd19-48d0-8c10-0f26029db59b-config-volume\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.766587    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:23:55.266567136 +0000 UTC m=+36.550617658 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766612    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8791898b-dc06-4841-8080-26b5d949fdaf-tmp\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.766666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766646    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e234fbf6-e9ba-410d-94c2-b780b451b92d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766690    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766724    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766734    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766756    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.766782    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.767346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.767190    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2722582d-cd19-48d0-8c10-0f26029db59b-config-volume\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.767815    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e234fbf6-e9ba-410d-94c2-b780b451b92d-additional-service-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.768000    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.768124    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.768140    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:54.768235    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:55.268217709 +0000 UTC m=+36.552268239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:23:54.768547 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.768363    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8791898b-dc06-4841-8080-26b5d949fdaf-tmp\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.768913 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.768687    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.769001 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.768976    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2722582d-cd19-48d0-8c10-0f26029db59b-tmp-dir\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.771756 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.771727    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.771856 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.771733    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.771856 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.771802    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.771856 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.771807    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.772252 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.772197    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e234fbf6-e9ba-410d-94c2-b780b451b92d-ca\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.772252 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.772204    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.772592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.772571    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/78151588-e645-407d-ab18-f0e5870972f6-cert\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.773283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.773253    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.773851 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.773829    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8791898b-dc06-4841-8080-26b5d949fdaf-klusterlet-config\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.776485 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.776439    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxxfz\" (UniqueName: \"kubernetes.io/projected/8791898b-dc06-4841-8080-26b5d949fdaf-kube-api-access-rxxfz\") pod \"klusterlet-addon-workmgr-548b979fb8-mb2sd\" (UID: \"8791898b-dc06-4841-8080-26b5d949fdaf\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.776636 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.776617    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp4zg\" (UniqueName: \"kubernetes.io/projected/78151588-e645-407d-ab18-f0e5870972f6-kube-api-access-gp4zg\") pod \"ingress-canary-dsztb\" (UID: \"78151588-e645-407d-ab18-f0e5870972f6\") " pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:54.777732 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.777265    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thbdw\" (UniqueName: \"kubernetes.io/projected/41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96-kube-api-access-thbdw\") pod \"managed-serviceaccount-addon-agent-85b97bb466-tpdgn\" (UID: \"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.777732 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.777695    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z7rkm\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.777732 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.777710    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrhv7\" (UniqueName: \"kubernetes.io/projected/2722582d-cd19-48d0-8c10-0f26029db59b-kube-api-access-vrhv7\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:54.779913 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.779876    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:54.780416 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.780397    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9d2z\" (UniqueName: \"kubernetes.io/projected/e234fbf6-e9ba-410d-94c2-b780b451b92d-kube-api-access-t9d2z\") pod \"cluster-proxy-proxy-agent-bcdc87884-hqdwd\" (UID: \"e234fbf6-e9ba-410d-94c2-b780b451b92d\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.865587 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.865495    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"
Jul 06 20:23:54.873602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.873576    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:23:54.901063 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.900755    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:23:54.904825 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:54.904647    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-dsztb"
Jul 06 20:23:55.065462 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.065432    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn"]
Jul 06 20:23:55.066590 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.066566    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"]
Jul 06 20:23:55.069964 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:55.069939    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod41f068b7_d7a1_4d0c_bd2c_e4e57aaf2b96.slice/crio-479415c06cf02f2270adfc7de929956ae7bf713ac5f2902e03f290c6c52f4682 WatchSource:0}: Error finding container 479415c06cf02f2270adfc7de929956ae7bf713ac5f2902e03f290c6c52f4682: Status 404 returned error can't find the container with id 479415c06cf02f2270adfc7de929956ae7bf713ac5f2902e03f290c6c52f4682
Jul 06 20:23:55.070136 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:55.070116    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode234fbf6_e9ba_410d_94c2_b780b451b92d.slice/crio-a16bdb8bf66124b40112eb03b190953a6d49a55bec66e90fc31a9671228d7895 WatchSource:0}: Error finding container a16bdb8bf66124b40112eb03b190953a6d49a55bec66e90fc31a9671228d7895: Status 404 returned error can't find the container with id a16bdb8bf66124b40112eb03b190953a6d49a55bec66e90fc31a9671228d7895
Jul 06 20:23:55.076459 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.076438    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-dsztb"]
Jul 06 20:23:55.079358 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:55.079338    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78151588_e645_407d_ab18_f0e5870972f6.slice/crio-9d1daae9e55226390d93f5e672788ac1396a38fd4158c8f33b3440dcedc5a07c WatchSource:0}: Error finding container 9d1daae9e55226390d93f5e672788ac1396a38fd4158c8f33b3440dcedc5a07c: Status 404 returned error can't find the container with id 9d1daae9e55226390d93f5e672788ac1396a38fd4158c8f33b3440dcedc5a07c
Jul 06 20:23:55.080742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.080722    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"]
Jul 06 20:23:55.083520 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:23:55.083502    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8791898b_dc06_4841_8080_26b5d949fdaf.slice/crio-9490e1289f24c879902cfd9e74c151507dd8979ce8126d900874c671b3498396 WatchSource:0}: Error finding container 9490e1289f24c879902cfd9e74c151507dd8979ce8126d900874c671b3498396: Status 404 returned error can't find the container with id 9490e1289f24c879902cfd9e74c151507dd8979ce8126d900874c671b3498396
Jul 06 20:23:55.270820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.270786    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:55.270993 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.270835    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:55.270993 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:55.270963    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:23:55.271113 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:55.271002    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:23:55.271113 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:55.271019    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:23:55.271113 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:55.271040    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:23:56.271020653 +0000 UTC m=+37.555071178 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:23:55.271113 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:55.271062    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:56.271048699 +0000 UTC m=+37.555099217 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:23:55.521891 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.521769    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd" event={"ID":"8791898b-dc06-4841-8080-26b5d949fdaf","Type":"ContainerStarted","Data":"9490e1289f24c879902cfd9e74c151507dd8979ce8126d900874c671b3498396"}
Jul 06 20:23:55.522998 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.522969    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn" event={"ID":"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96","Type":"ContainerStarted","Data":"479415c06cf02f2270adfc7de929956ae7bf713ac5f2902e03f290c6c52f4682"}
Jul 06 20:23:55.524237 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.524209    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dsztb" event={"ID":"78151588-e645-407d-ab18-f0e5870972f6","Type":"ContainerStarted","Data":"9d1daae9e55226390d93f5e672788ac1396a38fd4158c8f33b3440dcedc5a07c"}
Jul 06 20:23:55.525359 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:55.525334    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerStarted","Data":"a16bdb8bf66124b40112eb03b190953a6d49a55bec66e90fc31a9671228d7895"}
Jul 06 20:23:56.276508 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.276473    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:23:56.277056 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.276959    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:23:56.277439 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.277422    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:23:56.281269 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.281235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:56.281369 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.281299    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:56.281433 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:56.281414    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:23:56.281433 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:56.281427    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:23:56.281582 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:56.281479    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:23:58.281462997 +0000 UTC m=+39.565513519 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:23:56.281582 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.281498    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 20:23:56.281717 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:56.281591    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:23:56.281717 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:56.281628    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:23:58.28161698 +0000 UTC m=+39.565667502 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:23:56.282563 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.282119    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 20:23:56.282563 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.282455    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 20:23:56.282709 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.282666    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z7zd8\""
Jul 06 20:23:56.283059 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.282902    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 20:23:56.283355 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:56.282351    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hhd9j\""
Jul 06 20:23:57.901075 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:57.900926    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.070402 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.070272    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.140784 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.140686    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.251762 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.251627    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.304458 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.304367    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:23:58.304660 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:58.304547    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:23:58.304660 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:58.304616    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:24:02.304595575 +0000 UTC m=+43.588646096 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:23:58.305064 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.305039    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:23:58.305199 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:58.305186    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:23:58.305256 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:58.305204    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:23:58.305256 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:23:58.305245    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:02.305231595 +0000 UTC m=+43.589282118 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:23:58.359068 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.358972    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.493513 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.493427    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.594678 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.594637    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.828092 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.828065    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:58.893042 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:58.892950    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:59.246591 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:59.246502    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:23:59.329726 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:23:59.329687    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:00.037274 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:00.037192    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:00.107842 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:00.107804    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:00.420252 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:00.420213    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:00.484695 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:00.484665    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:02.340562 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:02.340504    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:02.340596    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:02.340640    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:02.340701    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:02.340713    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:02.340716    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:24:10.340696581 +0000 UTC m=+51.624747104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:24:02.341010 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:02.340746    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:10.340736484 +0000 UTC m=+51.624787004 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:24:04.552031 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.551809    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-dsztb" event={"ID":"78151588-e645-407d-ab18-f0e5870972f6","Type":"ContainerStarted","Data":"6de47161524f92e4dba4c5c6be5dac212b61251e2aec1cd74e08381741be0414"}
Jul 06 20:24:04.553960 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.553933    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerStarted","Data":"d3ec3c7ddcd78f8e976748cd5ddc0726d35755d54f0578557c1d877edce2e21d"}
Jul 06 20:24:04.554093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.553975    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerStarted","Data":"d7f2e9ae585bab38fc8e1c3d33b920bfaccd14756b509db2734105f9c36b1419"}
Jul 06 20:24:04.554093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.553998    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerStarted","Data":"d6d2859eed79a7582e57202e93586fed2e1455dc6bace018ca3006d1ad45912e"}
Jul 06 20:24:04.555417 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.555391    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd" event={"ID":"8791898b-dc06-4841-8080-26b5d949fdaf","Type":"ContainerStarted","Data":"cbb234b3dc1d756dff416c4e79fa6942b2b1b895a78195c3be341d660407dbc3"}
Jul 06 20:24:04.555583 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.555566    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:24:04.557006 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.556975    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn" event={"ID":"41f068b7-d7a1-4d0c-bd2c-e4e57aaf2b96","Type":"ContainerStarted","Data":"15f55c5ec2ba6188fd927af8615db42f7e8ba999981be5bac13b0bb6e493a575"}
Jul 06 20:24:04.557502 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.557485    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd"
Jul 06 20:24:04.559763 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.559742    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="1c3b258fa61eeab59d391c0a64d28aeb9f3660b39131ec98d5bf54a631d384bc" exitCode=0
Jul 06 20:24:04.559854 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.559773    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"1c3b258fa61eeab59d391c0a64d28aeb9f3660b39131ec98d5bf54a631d384bc"}
Jul 06 20:24:04.572186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.570064    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-dsztb" podStartSLOduration=1.650678396 podStartE2EDuration="10.570046936s" totalImagesPullingTime="8.91936854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:04.568674384 +0000 UTC m=+45.852724925" watchObservedRunningTime="2026-07-06 20:24:04.570046936 +0000 UTC m=+45.854097478"
Jul 06 20:24:04.609942 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.609834    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-548b979fb8-mb2sd" podStartSLOduration=28.687158155 podStartE2EDuration="37.609814481s" totalImagesPullingTime="8.922656326s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:04.60890901 +0000 UTC m=+45.892959552" watchObservedRunningTime="2026-07-06 20:24:04.609814481 +0000 UTC m=+45.893865021"
Jul 06 20:24:04.626271 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.626203    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-85b97bb466-tpdgn" podStartSLOduration=28.697473114 podStartE2EDuration="37.626178503s" totalImagesPullingTime="8.928705389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:04.625258598 +0000 UTC m=+45.909309140" watchObservedRunningTime="2026-07-06 20:24:04.626178503 +0000 UTC m=+45.910229046"
Jul 06 20:24:04.645741 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:04.645679    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" podStartSLOduration=28.701436689 podStartE2EDuration="37.645655424s" totalImagesPullingTime="8.944218735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:04.645380209 +0000 UTC m=+45.929430750" watchObservedRunningTime="2026-07-06 20:24:04.645655424 +0000 UTC m=+45.929705968"
Jul 06 20:24:05.548258 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.548222    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:05.566089 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.566057    2554 generic.go:356] "Generic (PLEG): container finished" podID="4799b32f-18c9-450e-8724-7cba6bc1bdf1" containerID="412b3fe0ca055432c4f9ef3a18c05469b24ec8f5feb190190369359e2781e537" exitCode=0
Jul 06 20:24:05.566559 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.566138    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerDied","Data":"412b3fe0ca055432c4f9ef3a18c05469b24ec8f5feb190190369359e2781e537"}
Jul 06 20:24:05.610419 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.610388    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:05.942042 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.941961    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:05.994371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:05.994338    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:06.576091 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:06.576058    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" event={"ID":"4799b32f-18c9-450e-8724-7cba6bc1bdf1","Type":"ContainerStarted","Data":"f864164d101a2642b1c6374fbb303b1d65be9d9c2c5bd20acc2a8206249549c9"}
Jul 06 20:24:06.600107 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:06.600044    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-4pf8v" podStartSLOduration=8.259044808 podStartE2EDuration="47.600023087s" totalImagesPullingTime="39.340978279s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:06.598725492 +0000 UTC m=+47.882776030" watchObservedRunningTime="2026-07-06 20:24:06.600023087 +0000 UTC m=+47.884073629"
Jul 06 20:24:06.721210 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:06.721176    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:06.821798 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:06.821767    2554 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log" limit=1024
Jul 06 20:24:10.405131 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:10.405099    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:10.405150    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:10.405242    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:10.405258    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:10.405243    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:10.405310    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:26.405297166 +0000 UTC m=+67.689347689 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:24:10.405574 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:10.405322    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:24:26.405316363 +0000 UTC m=+67.689366882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:24:21.522780 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:21.522754    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-59hxf"
Jul 06 20:24:23.912399 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:23.912351    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:24:23.915024 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:23.915006    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 06 20:24:23.923004 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:23.922980    2554 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Jul 06 20:24:23.923074 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:23.923061    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs podName:8dce919a-95e0-4f51-bd85-053330aa2d72 nodeName:}" failed. No retries permitted until 2026-07-06 20:25:27.923044107 +0000 UTC m=+129.207094626 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs") pod "network-metrics-daemon-k74c5" (UID: "8dce919a-95e0-4f51-bd85-053330aa2d72") : secret "metrics-daemon-secret" not found
Jul 06 20:24:24.012745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.012705    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:24:24.012745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.012749    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:24:24.015249 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.015222    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 06 20:24:24.015380 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.015362    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 06 20:24:24.025173 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.025156    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 06 20:24:24.026672 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.026644    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/59ae90f0-b70b-4374-90ac-d84276e8bccd-original-pull-secret\") pod \"global-pull-secret-syncer-xr6rj\" (UID: \"59ae90f0-b70b-4374-90ac-d84276e8bccd\") " pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:24:24.035166 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.035143    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcv5r\" (UniqueName: \"kubernetes.io/projected/d2d04775-be12-4888-a7a4-ac81c6e5bfdb-kube-api-access-mcv5r\") pod \"network-check-target-57bm6\" (UID: \"d2d04775-be12-4888-a7a4-ac81c6e5bfdb\") " pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:24:24.206493 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.206407    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-xr6rj"
Jul 06 20:24:24.216290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.216267    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hhd9j\""
Jul 06 20:24:24.224261 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.224229    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:24:24.340416 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.340385    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-xr6rj"]
Jul 06 20:24:24.343956 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:24:24.343931    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ae90f0_b70b_4374_90ac_d84276e8bccd.slice/crio-bf15fcd4d1b56582fbe52ef7677386738536aeb77cb594bfc3c684f88d13ae48 WatchSource:0}: Error finding container bf15fcd4d1b56582fbe52ef7677386738536aeb77cb594bfc3c684f88d13ae48: Status 404 returned error can't find the container with id bf15fcd4d1b56582fbe52ef7677386738536aeb77cb594bfc3c684f88d13ae48
Jul 06 20:24:24.357044 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.357021    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-57bm6"]
Jul 06 20:24:24.360153 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:24:24.360131    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2d04775_be12_4888_a7a4_ac81c6e5bfdb.slice/crio-bc1f38c2829e5b64419351f7c9b3be5312554fd55c8009e0e8610d49c49b0c8f WatchSource:0}: Error finding container bc1f38c2829e5b64419351f7c9b3be5312554fd55c8009e0e8610d49c49b0c8f: Status 404 returned error can't find the container with id bc1f38c2829e5b64419351f7c9b3be5312554fd55c8009e0e8610d49c49b0c8f
Jul 06 20:24:24.629424 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.629388    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xr6rj" event={"ID":"59ae90f0-b70b-4374-90ac-d84276e8bccd","Type":"ContainerStarted","Data":"bf15fcd4d1b56582fbe52ef7677386738536aeb77cb594bfc3c684f88d13ae48"}
Jul 06 20:24:24.630482 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:24.630447    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57bm6" event={"ID":"d2d04775-be12-4888-a7a4-ac81c6e5bfdb","Type":"ContainerStarted","Data":"bc1f38c2829e5b64419351f7c9b3be5312554fd55c8009e0e8610d49c49b0c8f"}
Jul 06 20:24:26.432875 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:26.432832    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:26.432909    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:26.433004    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:26.433019    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:26.433034    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:26.433083    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:24:58.433063864 +0000 UTC m=+99.717114387 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:24:26.433305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:26.433102    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:24:58.433092817 +0000 UTC m=+99.717143340 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:24:30.651492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:30.651456    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-57bm6" event={"ID":"d2d04775-be12-4888-a7a4-ac81c6e5bfdb","Type":"ContainerStarted","Data":"d861f12f6abcb34eabc7074d5b9f4fe561e8bbad306fb27a9c815ae81dde0f33"}
Jul 06 20:24:30.651983 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:30.651562    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:24:30.652915 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:30.652892    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-xr6rj" event={"ID":"59ae90f0-b70b-4374-90ac-d84276e8bccd","Type":"ContainerStarted","Data":"28e38cafe792e1e28ce3064ee957eef1ff652ce10fa9a18fd02ed4938648008d"}
Jul 06 20:24:30.669239 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:30.669195    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-57bm6" podStartSLOduration=65.841556053 podStartE2EDuration="1m11.669178598s" totalImagesPullingTime="5.827622545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:30.668558814 +0000 UTC m=+71.952609352" watchObservedRunningTime="2026-07-06 20:24:30.669178598 +0000 UTC m=+71.953229139"
Jul 06 20:24:30.683866 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:30.683810    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-xr6rj" podStartSLOduration=64.836319082 podStartE2EDuration="1m10.683792123s" totalImagesPullingTime="5.847473041s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:24:30.682874674 +0000 UTC m=+71.966925215" watchObservedRunningTime="2026-07-06 20:24:30.683792123 +0000 UTC m=+71.967842665"
Jul 06 20:24:58.476726 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:58.476686    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:24:58.476743    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") pod \"image-registry-5f6fbdd4dd-m5ns4\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") " pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:58.476843    2554 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:58.476859    2554 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4: secret "image-registry-tls" not found
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:58.476905    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls podName:7d850d28-4eed-469c-b848-0ec069d4e939 nodeName:}" failed. No retries permitted until 2026-07-06 20:26:02.476892016 +0000 UTC m=+163.760942539 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls") pod "image-registry-5f6fbdd4dd-m5ns4" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939") : secret "image-registry-tls" not found
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:58.476842    2554 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 06 20:24:58.477223 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:24:58.476996    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls podName:2722582d-cd19-48d0-8c10-0f26029db59b nodeName:}" failed. No retries permitted until 2026-07-06 20:26:02.476980486 +0000 UTC m=+163.761031008 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls") pod "dns-default-cfjph" (UID: "2722582d-cd19-48d0-8c10-0f26029db59b") : secret "dns-default-metrics-tls" not found
Jul 06 20:25:00.777659 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.777621    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-pgdml"]
Jul 06 20:25:00.809745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.809718    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgdml"]
Jul 06 20:25:00.809895 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.809839    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.812230 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.812212    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 06 20:25:00.812516 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.812502    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 06 20:25:00.813034 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.813014    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 06 20:25:00.813142 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.813084    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 06 20:25:00.813142 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.813091    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6n8v\""
Jul 06 20:25:00.893893 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.893851    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.894058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.893915    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.894058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.893945    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1b98fc0-444e-4221-92d9-0eeed9591a1c-crio-socket\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.894058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.893972    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b98fc0-444e-4221-92d9-0eeed9591a1c-data-volume\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.894058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.893989    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wp7x\" (UniqueName: \"kubernetes.io/projected/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-api-access-2wp7x\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.994920 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.994882    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995096 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.994965    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995096 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.994997    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1b98fc0-444e-4221-92d9-0eeed9591a1c-crio-socket\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995096 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.995028    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b98fc0-444e-4221-92d9-0eeed9591a1c-data-volume\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995096 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:00.995049    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:00.995269 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:00.995099    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:25:00.995269 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:00.995117    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:01.4950977 +0000 UTC m=+102.779148239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:00.995269 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.995113    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f1b98fc0-444e-4221-92d9-0eeed9591a1c-crio-socket\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995269 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.995050    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wp7x\" (UniqueName: \"kubernetes.io/projected/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-api-access-2wp7x\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:00.995269 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:00.995203    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:01.495185328 +0000 UTC m=+102.779235878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : configmap "kube-rbac-proxy" not found
Jul 06 20:25:00.995440 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:00.995381    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f1b98fc0-444e-4221-92d9-0eeed9591a1c-data-volume\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:01.005592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:01.005568    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wp7x\" (UniqueName: \"kubernetes.io/projected/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-api-access-2wp7x\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:01.499981 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:01.499939    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:01.500161 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:01.499998    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:01.500161 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:01.500082    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:25:01.500161 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:01.500092    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:01.500161 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:01.500148    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:02.50013557 +0000 UTC m=+103.784186089 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : configmap "kube-rbac-proxy" not found
Jul 06 20:25:01.500161 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:01.500161    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:02.50015482 +0000 UTC m=+103.784205338 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:01.659195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:01.659167    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-57bm6"
Jul 06 20:25:02.508273 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:02.508235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:02.508759 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:02.508304    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:02.508759 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:02.508363    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:25:02.508759 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:02.508420    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:02.508759 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:02.508442    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:04.508425756 +0000 UTC m=+105.792476275 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : configmap "kube-rbac-proxy" not found
Jul 06 20:25:02.508759 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:02.508485    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:04.508468393 +0000 UTC m=+105.792518911 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:02.947540 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:02.947512    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rm27j_1004a321-ebe7-4838-88bf-6c0b45512844/dns-node-resolver/0.log"
Jul 06 20:25:03.383812 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.383780    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"]
Jul 06 20:25:03.401549 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.401503    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"]
Jul 06 20:25:03.401722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.401672    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"
Jul 06 20:25:03.403893 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.403874    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-pk9cf\""
Jul 06 20:25:03.517319 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.517278    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d8sc\" (UniqueName: \"kubernetes.io/projected/922e0ba1-4dfe-453a-ba7a-0cdf56112ff5-kube-api-access-7d8sc\") pod \"network-check-source-989f8b8dc-k2n9f\" (UID: \"922e0ba1-4dfe-453a-ba7a-0cdf56112ff5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"
Jul 06 20:25:03.618303 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.618255    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d8sc\" (UniqueName: \"kubernetes.io/projected/922e0ba1-4dfe-453a-ba7a-0cdf56112ff5-kube-api-access-7d8sc\") pod \"network-check-source-989f8b8dc-k2n9f\" (UID: \"922e0ba1-4dfe-453a-ba7a-0cdf56112ff5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"
Jul 06 20:25:03.626894 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.626862    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d8sc\" (UniqueName: \"kubernetes.io/projected/922e0ba1-4dfe-453a-ba7a-0cdf56112ff5-kube-api-access-7d8sc\") pod \"network-check-source-989f8b8dc-k2n9f\" (UID: \"922e0ba1-4dfe-453a-ba7a-0cdf56112ff5\") " pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"
Jul 06 20:25:03.710361 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.710264    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"
Jul 06 20:25:03.746818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.746786    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mjjfd_f616263c-0395-4994-95de-ec5860eac4a7/node-ca/0.log"
Jul 06 20:25:03.834003 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:03.833964    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f"]
Jul 06 20:25:03.840029 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:25:03.839990    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod922e0ba1_4dfe_453a_ba7a_0cdf56112ff5.slice/crio-06a2af2aff7f2ab11a9c1311daab9fc81eb8f72d51b9590754eabeeb2396c5b9 WatchSource:0}: Error finding container 06a2af2aff7f2ab11a9c1311daab9fc81eb8f72d51b9590754eabeeb2396c5b9: Status 404 returned error can't find the container with id 06a2af2aff7f2ab11a9c1311daab9fc81eb8f72d51b9590754eabeeb2396c5b9
Jul 06 20:25:04.347306 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.347255    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dsztb_78151588-e645-407d-ab18-f0e5870972f6/serve-healthcheck-canary/0.log"
Jul 06 20:25:04.524499 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.524447    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:04.524906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.524555    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:04.524906 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:04.524638    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:04.524906 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:04.524673    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:25:04.524906 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:04.524704    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:08.524687598 +0000 UTC m=+109.808738125 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:04.524906 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:04.524728    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:08.52471418 +0000 UTC m=+109.808764709 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : configmap "kube-rbac-proxy" not found
Jul 06 20:25:04.753443 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.753358    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f" event={"ID":"922e0ba1-4dfe-453a-ba7a-0cdf56112ff5","Type":"ContainerStarted","Data":"4ae9d50718b1e818682f43f6f91037424071b8efa337de3c0cc960bd89011b4b"}
Jul 06 20:25:04.753443 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.753407    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f" event={"ID":"922e0ba1-4dfe-453a-ba7a-0cdf56112ff5","Type":"ContainerStarted","Data":"06a2af2aff7f2ab11a9c1311daab9fc81eb8f72d51b9590754eabeeb2396c5b9"}
Jul 06 20:25:04.781290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:04.781204    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-989f8b8dc-k2n9f" podStartSLOduration=1.781181927 podStartE2EDuration="1.781181927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:25:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:04.779940196 +0000 UTC m=+106.063990751" watchObservedRunningTime="2026-07-06 20:25:04.781181927 +0000 UTC m=+106.065232470"
Jul 06 20:25:08.559806 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:08.559768    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:08.560216 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:08.559844    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:08.560216 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:08.559886    2554 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 06 20:25:08.560216 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:08.559940    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:08.560216 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:08.559947    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:16.559932749 +0000 UTC m=+117.843983267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : configmap "kube-rbac-proxy" not found
Jul 06 20:25:08.560216 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:08.559983    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:16.559971748 +0000 UTC m=+117.844022266 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:16.623177 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:16.623135    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:16.623686 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:16.623327    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:16.623686 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:16.623450    2554 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:16.623686 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:16.623522    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls podName:f1b98fc0-444e-4221-92d9-0eeed9591a1c nodeName:}" failed. No retries permitted until 2026-07-06 20:25:32.623503028 +0000 UTC m=+133.907553570 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-pgdml" (UID: "f1b98fc0-444e-4221-92d9-0eeed9591a1c") : secret "insights-runtime-extractor-tls" not found
Jul 06 20:25:16.623850 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:16.623709    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f1b98fc0-444e-4221-92d9-0eeed9591a1c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:28.006844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.006790    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:25:28.009113 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.009089    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8dce919a-95e0-4f51-bd85-053330aa2d72-metrics-certs\") pod \"network-metrics-daemon-k74c5\" (UID: \"8dce919a-95e0-4f51-bd85-053330aa2d72\") " pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:25:28.097053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.097022    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-z7zd8\""
Jul 06 20:25:28.105037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.105016    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-k74c5"
Jul 06 20:25:28.227212 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.227179    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-k74c5"]
Jul 06 20:25:28.229894 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:25:28.229858    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dce919a_95e0_4f51_bd85_053330aa2d72.slice/crio-0237466f431fffe9d5fe714dec3c4c915df8b3ece7f3d11ec677c7848cf1fca3 WatchSource:0}: Error finding container 0237466f431fffe9d5fe714dec3c4c915df8b3ece7f3d11ec677c7848cf1fca3: Status 404 returned error can't find the container with id 0237466f431fffe9d5fe714dec3c4c915df8b3ece7f3d11ec677c7848cf1fca3
Jul 06 20:25:28.825791 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:28.825754    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k74c5" event={"ID":"8dce919a-95e0-4f51-bd85-053330aa2d72","Type":"ContainerStarted","Data":"0237466f431fffe9d5fe714dec3c4c915df8b3ece7f3d11ec677c7848cf1fca3"}
Jul 06 20:25:29.830340 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:29.830304    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k74c5" event={"ID":"8dce919a-95e0-4f51-bd85-053330aa2d72","Type":"ContainerStarted","Data":"319f32bc9953ef0d622032b51bae86f3b3d22c12d7a9bf5b718d6740cd6c7931"}
Jul 06 20:25:29.830781 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:29.830394    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-k74c5" event={"ID":"8dce919a-95e0-4f51-bd85-053330aa2d72","Type":"ContainerStarted","Data":"0a5320a8aec3c4953cad20dc106e62d2ccf257bb782664b3ff074194e55264d9"}
Jul 06 20:25:29.847786 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:29.847735    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-k74c5" podStartSLOduration=129.581028254 podStartE2EDuration="2m10.847718628s" totalImagesPullingTime="1.266690374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:29.847519853 +0000 UTC m=+131.131570398" watchObservedRunningTime="2026-07-06 20:25:29.847718628 +0000 UTC m=+131.131769169"
Jul 06 20:25:32.643993 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:32.643959    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:32.646221 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:32.646202    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f1b98fc0-444e-4221-92d9-0eeed9591a1c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-pgdml\" (UID: \"f1b98fc0-444e-4221-92d9-0eeed9591a1c\") " pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:32.921658 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:32.921578    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6n8v\""
Jul 06 20:25:32.929039 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:32.929020    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-pgdml"
Jul 06 20:25:33.067229 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:33.067199    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-pgdml"]
Jul 06 20:25:33.071285 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:25:33.071252    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b98fc0_444e_4221_92d9_0eeed9591a1c.slice/crio-4eb83b7703b07c549361784196d4c757b91955975a3cdd47f72993a9edff9d8b WatchSource:0}: Error finding container 4eb83b7703b07c549361784196d4c757b91955975a3cdd47f72993a9edff9d8b: Status 404 returned error can't find the container with id 4eb83b7703b07c549361784196d4c757b91955975a3cdd47f72993a9edff9d8b
Jul 06 20:25:33.845324 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:33.845286    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgdml" event={"ID":"f1b98fc0-444e-4221-92d9-0eeed9591a1c","Type":"ContainerStarted","Data":"4eb83b7703b07c549361784196d4c757b91955975a3cdd47f72993a9edff9d8b"}
Jul 06 20:25:34.878718 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:34.876124    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" podUID="e234fbf6-e9ba-410d-94c2-b780b451b92d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 20:25:36.857510 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:36.857470    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgdml" event={"ID":"f1b98fc0-444e-4221-92d9-0eeed9591a1c","Type":"ContainerStarted","Data":"847af910427e7cd9fb69161c313da903f347afff8d68ab16b13a44dc7c4d60a9"}
Jul 06 20:25:37.863440 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:37.863404    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgdml" event={"ID":"f1b98fc0-444e-4221-92d9-0eeed9591a1c","Type":"ContainerStarted","Data":"b0d8bf9db1340951e7015b83e66b61d1cf66121b2c69094cc2936c8a87972af8"}
Jul 06 20:25:39.870879 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:39.870845    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-pgdml" event={"ID":"f1b98fc0-444e-4221-92d9-0eeed9591a1c","Type":"ContainerStarted","Data":"19d0a23edbddd329ac6d5f6de82a122b9024a969d56d39bb979fd52401238206"}
Jul 06 20:25:39.892210 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:39.892160    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-pgdml" podStartSLOduration=33.291419775 podStartE2EDuration="39.892145272s" totalImagesPullingTime="6.600725497s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 20:25:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:39.891160484 +0000 UTC m=+141.175211026" watchObservedRunningTime="2026-07-06 20:25:39.892145272 +0000 UTC m=+141.176195814"
Jul 06 20:25:44.875085 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:44.875045    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" podUID="e234fbf6-e9ba-410d-94c2-b780b451b92d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 20:25:45.143260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.143166    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mfcbb"]
Jul 06 20:25:45.145579 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.145555    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.150550 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.150509    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 06 20:25:45.150812 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.150764    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 06 20:25:45.150944 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.150846    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 06 20:25:45.151023 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.150951    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 06 20:25:45.151259 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.151240    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-h2cwn\""
Jul 06 20:25:45.151363 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.151247    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 06 20:25:45.151363 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.151308    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 06 20:25:45.244076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244037    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-root\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244071    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls7xr\" (UniqueName: \"kubernetes.io/projected/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-kube-api-access-ls7xr\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244284 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244090    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-wtmp\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244284 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244123    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-metrics-client-ca\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244284 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244190    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-sys\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244284 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244227    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-textfile\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244284 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244245    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244437 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244323    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.244437 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.244356    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345449 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345419    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-sys\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345635 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345470    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-textfile\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345635 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345488    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345635 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345518    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345635 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345557    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-sys\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345635 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345565    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345657    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-root\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345687    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls7xr\" (UniqueName: \"kubernetes.io/projected/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-kube-api-access-ls7xr\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345719    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-wtmp\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345733    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-root\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:45.345687    2554 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345765    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-metrics-client-ca\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:45.345815    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls podName:3b5cb8f1-e084-4148-95fd-8c4702c4a05f nodeName:}" failed. No retries permitted until 2026-07-06 20:25:45.84579445 +0000 UTC m=+147.129844978 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls") pod "node-exporter-mfcbb" (UID: "3b5cb8f1-e084-4148-95fd-8c4702c4a05f") : secret "node-exporter-tls" not found
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345862    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-wtmp\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.345918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.345870    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-textfile\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.346260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.346204    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-accelerators-collector-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.346301 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.346269    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-metrics-client-ca\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.348021 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.348000    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.354242 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.354218    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls7xr\" (UniqueName: \"kubernetes.io/projected/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-kube-api-access-ls7xr\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.849819 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.849772    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:45.852143 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:45.852122    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3b5cb8f1-e084-4148-95fd-8c4702c4a05f-node-exporter-tls\") pod \"node-exporter-mfcbb\" (UID: \"3b5cb8f1-e084-4148-95fd-8c4702c4a05f\") " pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:46.054315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:46.054280    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mfcbb"
Jul 06 20:25:46.063968 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:25:46.063939    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b5cb8f1_e084_4148_95fd_8c4702c4a05f.slice/crio-231b219e5f14265d16cf4c611694e515fda6e898d915e04c1e5d3b09eb5f38bf WatchSource:0}: Error finding container 231b219e5f14265d16cf4c611694e515fda6e898d915e04c1e5d3b09eb5f38bf: Status 404 returned error can't find the container with id 231b219e5f14265d16cf4c611694e515fda6e898d915e04c1e5d3b09eb5f38bf
Jul 06 20:25:46.893926 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:46.893894    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfcbb" event={"ID":"3b5cb8f1-e084-4148-95fd-8c4702c4a05f","Type":"ContainerStarted","Data":"231b219e5f14265d16cf4c611694e515fda6e898d915e04c1e5d3b09eb5f38bf"}
Jul 06 20:25:47.898720 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:47.898684    2554 generic.go:356] "Generic (PLEG): container finished" podID="3b5cb8f1-e084-4148-95fd-8c4702c4a05f" containerID="0ba7cb3b43211f4d17f12136396a11a312fe4b98e0a6441941be858b9624f139" exitCode=0
Jul 06 20:25:47.899183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:47.898738    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfcbb" event={"ID":"3b5cb8f1-e084-4148-95fd-8c4702c4a05f","Type":"ContainerDied","Data":"0ba7cb3b43211f4d17f12136396a11a312fe4b98e0a6441941be858b9624f139"}
Jul 06 20:25:48.903654 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:48.903616    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfcbb" event={"ID":"3b5cb8f1-e084-4148-95fd-8c4702c4a05f","Type":"ContainerStarted","Data":"821350a9ccd2d67f7dec8499ba1d1fc0c86cbf7397e3bf391676282f8387e940"}
Jul 06 20:25:48.904178 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:48.903673    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mfcbb" event={"ID":"3b5cb8f1-e084-4148-95fd-8c4702c4a05f","Type":"ContainerStarted","Data":"df18e2f276ee2485e26e6c552beab89ef75ef31f2f0d2da9ed4851a038c65d27"}
Jul 06 20:25:48.925889 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:48.925843    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mfcbb" podStartSLOduration=2.680377339 podStartE2EDuration="3.925824731s" totalImagesPullingTime="1.179838329s" totalInitContainerRuntime="65.609063ms" isStatefulPod=true podCreationTimestamp="2026-07-06 20:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:25:48.924195695 +0000 UTC m=+150.208246247" watchObservedRunningTime="2026-07-06 20:25:48.925824731 +0000 UTC m=+150.209875271"
Jul 06 20:25:54.875396 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:54.875355    2554 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" podUID="e234fbf6-e9ba-410d-94c2-b780b451b92d" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Jul 06 20:25:54.875778 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:54.875436    2554 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd"
Jul 06 20:25:54.876023 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:54.875991    2554 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"d3ec3c7ddcd78f8e976748cd5ddc0726d35755d54f0578557c1d877edce2e21d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Jul 06 20:25:54.876069 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:54.876045    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" podUID="e234fbf6-e9ba-410d-94c2-b780b451b92d" containerName="service-proxy" containerID="cri-o://d3ec3c7ddcd78f8e976748cd5ddc0726d35755d54f0578557c1d877edce2e21d" gracePeriod=30
Jul 06 20:25:55.677599 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.677565    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"]
Jul 06 20:25:55.677832 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:55.677814    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4" podUID="7d850d28-4eed-469c-b848-0ec069d4e939"
Jul 06 20:25:55.927945 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.927863    2554 generic.go:356] "Generic (PLEG): container finished" podID="e234fbf6-e9ba-410d-94c2-b780b451b92d" containerID="d3ec3c7ddcd78f8e976748cd5ddc0726d35755d54f0578557c1d877edce2e21d" exitCode=2
Jul 06 20:25:55.927945 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.927928    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerDied","Data":"d3ec3c7ddcd78f8e976748cd5ddc0726d35755d54f0578557c1d877edce2e21d"}
Jul 06 20:25:55.928359 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.927949    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:25:55.928359 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.927968    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-bcdc87884-hqdwd" event={"ID":"e234fbf6-e9ba-410d-94c2-b780b451b92d","Type":"ContainerStarted","Data":"8fc71c4131c7694be2bab528dd3bf47406f08d5754f42870af8925b5d13c87a1"}
Jul 06 20:25:55.933275 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:55.933251    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:25:56.033410 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033373    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z7rkm\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033449    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033484    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033519    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033569    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033869 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033613    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.033869 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.033649    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca\") pod \"7d850d28-4eed-469c-b848-0ec069d4e939\" (UID: \"7d850d28-4eed-469c-b848-0ec069d4e939\") "
Jul 06 20:25:56.034163 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.034128    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:25:56.034325 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.034201    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:25:56.034506 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.034482    2554 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7d850d28-4eed-469c-b848-0ec069d4e939-ca-trust-extracted\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.034719 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.034511    2554 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-trusted-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.034719 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.034672    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:25:56.036042 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.036017    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm" (OuterVolumeSpecName: "kube-api-access-z7rkm") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "kube-api-access-z7rkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:25:56.036138 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.036105    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:25:56.036208 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.036189    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:25:56.036290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.036265    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7d850d28-4eed-469c-b848-0ec069d4e939" (UID: "7d850d28-4eed-469c-b848-0ec069d4e939"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:25:56.135685 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.135643    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z7rkm\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-kube-api-access-z7rkm\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.135685 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.135677    2554 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-image-registry-private-configuration\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.135685 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.135691    2554 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7d850d28-4eed-469c-b848-0ec069d4e939-registry-certificates\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.135921 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.135705    2554 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-bound-sa-token\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.135921 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.135717    2554 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7d850d28-4eed-469c-b848-0ec069d4e939-installation-pull-secrets\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:56.930731 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.930699    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"
Jul 06 20:25:56.967577 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.967522    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"]
Jul 06 20:25:56.971817 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:56.971786    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f6fbdd4dd-m5ns4"]
Jul 06 20:25:57.042319 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:57.042284    2554 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7d850d28-4eed-469c-b848-0ec069d4e939-registry-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:25:57.280630 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:57.280602    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d850d28-4eed-469c-b848-0ec069d4e939" path="/var/lib/kubelet/pods/7d850d28-4eed-469c-b848-0ec069d4e939/volumes"
Jul 06 20:25:57.587983 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:25:57.587886    2554 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-cfjph" podUID="2722582d-cd19-48d0-8c10-0f26029db59b"
Jul 06 20:25:57.933799 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:25:57.933719    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cfjph"
Jul 06 20:26:02.481385 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.481343    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:26:02.483758 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.483733    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2722582d-cd19-48d0-8c10-0f26029db59b-metrics-tls\") pod \"dns-default-cfjph\" (UID: \"2722582d-cd19-48d0-8c10-0f26029db59b\") " pod="openshift-dns/dns-default-cfjph"
Jul 06 20:26:02.737573 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.737460    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mqhkz\""
Jul 06 20:26:02.744937 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.744895    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-cfjph"
Jul 06 20:26:02.881436 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.881407    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-cfjph"]
Jul 06 20:26:02.884148 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:26:02.884117    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2722582d_cd19_48d0_8c10_0f26029db59b.slice/crio-691891e98a5f8b47563bf5fa04935e5a799dcde1d3c79c7ea6f89b332ac7f9d7 WatchSource:0}: Error finding container 691891e98a5f8b47563bf5fa04935e5a799dcde1d3c79c7ea6f89b332ac7f9d7: Status 404 returned error can't find the container with id 691891e98a5f8b47563bf5fa04935e5a799dcde1d3c79c7ea6f89b332ac7f9d7
Jul 06 20:26:02.951383 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:02.951351    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfjph" event={"ID":"2722582d-cd19-48d0-8c10-0f26029db59b","Type":"ContainerStarted","Data":"691891e98a5f8b47563bf5fa04935e5a799dcde1d3c79c7ea6f89b332ac7f9d7"}
Jul 06 20:26:04.963329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:04.963287    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfjph" event={"ID":"2722582d-cd19-48d0-8c10-0f26029db59b","Type":"ContainerStarted","Data":"056822d5091b907491653ddcd0b7e37bfb41c8cc51141401fb18b37008c30e86"}
Jul 06 20:26:05.968968 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:05.968932    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-cfjph" event={"ID":"2722582d-cd19-48d0-8c10-0f26029db59b","Type":"ContainerStarted","Data":"b0701f0e3429010701eff14cfe3688794a872850d64ea7d736f9345ade06438f"}
Jul 06 20:26:05.969450 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:05.969078    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-cfjph"
Jul 06 20:26:05.988335 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:05.988285    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-cfjph" podStartSLOduration=130.045803284 podStartE2EDuration="2m11.988268417s" totalImagesPullingTime="1.942465133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:23:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:26:05.987067225 +0000 UTC m=+167.271117806" watchObservedRunningTime="2026-07-06 20:26:05.988268417 +0000 UTC m=+167.272318958"
Jul 06 20:26:15.975137 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:26:15.975057    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-cfjph"
Jul 06 20:28:19.100728 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:28:19.100697    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:28:19.103808 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:28:19.103790    2554 kubelet.go:1648] "Image garbage collection succeeded"
Jul 06 20:28:19.106020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:28:19.105998    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:30:01.465592 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.465556    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"]
Jul 06 20:30:01.468580 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.468564    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.471396 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.471371    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Jul 06 20:30:01.472111 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.472091    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Jul 06 20:30:01.472237 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.472118    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-85w5x\""
Jul 06 20:30:01.484606 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.484584    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"]
Jul 06 20:30:01.613185 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.613144    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.613371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.613208    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22fjj\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-kube-api-access-22fjj\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.714251 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.714202    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22fjj\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-kube-api-access-22fjj\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.714251 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.714267    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.724709 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.724640    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.724825 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.724714    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22fjj\" (UniqueName: \"kubernetes.io/projected/d4b63a3e-94c8-41ab-800c-530ff640e801-kube-api-access-22fjj\") pod \"cert-manager-cainjector-559cdd8c87-6dv6b\" (UID: \"d4b63a3e-94c8-41ab-800c-530ff640e801\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.777259 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.777228    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"
Jul 06 20:30:01.902007 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.901975    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b"]
Jul 06 20:30:01.905487 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:30:01.905455    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd4b63a3e_94c8_41ab_800c_530ff640e801.slice/crio-77177bc148eb3dc49f720b11c00ef6eba872a098c2bcb148e9f3332aa6f292ac WatchSource:0}: Error finding container 77177bc148eb3dc49f720b11c00ef6eba872a098c2bcb148e9f3332aa6f292ac: Status 404 returned error can't find the container with id 77177bc148eb3dc49f720b11c00ef6eba872a098c2bcb148e9f3332aa6f292ac
Jul 06 20:30:01.907322 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:01.907305    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:30:02.723609 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:02.723575    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b" event={"ID":"d4b63a3e-94c8-41ab-800c-530ff640e801","Type":"ContainerStarted","Data":"77177bc148eb3dc49f720b11c00ef6eba872a098c2bcb148e9f3332aa6f292ac"}
Jul 06 20:30:07.742715 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:07.742675    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b" event={"ID":"d4b63a3e-94c8-41ab-800c-530ff640e801","Type":"ContainerStarted","Data":"b9e4baac7853385f7d5e12cd7a28860012665abc6f1461309b2bbeca4053bb00"}
Jul 06 20:30:07.760647 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:07.760600    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6dv6b" podStartSLOduration=1.7373084410000001 podStartE2EDuration="6.760584472s" totalImagesPullingTime="5.023276031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:30:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:30:07.759310064 +0000 UTC m=+409.043360628" watchObservedRunningTime="2026-07-06 20:30:07.760584472 +0000 UTC m=+409.044635013"
Jul 06 20:30:10.228733 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.228689    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"]
Jul 06 20:30:10.231704 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.231688    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.234243 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.234222    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 06 20:30:10.234406 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.234308    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-jzwfl\""
Jul 06 20:30:10.235156 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.235141    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 06 20:30:10.242136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.242115    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"]
Jul 06 20:30:10.378950 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.378908    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.379120 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.378959    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrdzl\" (UniqueName: \"kubernetes.io/projected/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-kube-api-access-hrdzl\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.479306 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.479241    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.479306 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.479285    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrdzl\" (UniqueName: \"kubernetes.io/projected/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-kube-api-access-hrdzl\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.479698 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.479675    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.487519 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.487494    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrdzl\" (UniqueName: \"kubernetes.io/projected/42dfa53d-03b2-4c6d-89b8-8e6140e7a301-kube-api-access-hrdzl\") pod \"openshift-lws-operator-fd8ccff4c-tt2gs\" (UID: \"42dfa53d-03b2-4c6d-89b8-8e6140e7a301\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.540819 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.540784    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"
Jul 06 20:30:10.664123 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.664092    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs"]
Jul 06 20:30:10.666765 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:30:10.666737    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod42dfa53d_03b2_4c6d_89b8_8e6140e7a301.slice/crio-bc8f514c7ae56f28eb9cb8eed5fe3330c94f1a6fea49390b11046484dc076460 WatchSource:0}: Error finding container bc8f514c7ae56f28eb9cb8eed5fe3330c94f1a6fea49390b11046484dc076460: Status 404 returned error can't find the container with id bc8f514c7ae56f28eb9cb8eed5fe3330c94f1a6fea49390b11046484dc076460
Jul 06 20:30:10.753674 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:10.753594    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs" event={"ID":"42dfa53d-03b2-4c6d-89b8-8e6140e7a301","Type":"ContainerStarted","Data":"bc8f514c7ae56f28eb9cb8eed5fe3330c94f1a6fea49390b11046484dc076460"}
Jul 06 20:30:13.767653 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:13.767606    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs" event={"ID":"42dfa53d-03b2-4c6d-89b8-8e6140e7a301","Type":"ContainerStarted","Data":"7a89bb50ab2652f5c8f9d7d4148e6f0d5e36de915433cf39c8fb4e9183d13b65"}
Jul 06 20:30:13.787871 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:13.787818    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tt2gs" podStartSLOduration=1.5285873589999999 podStartE2EDuration="3.787800909s" totalImagesPullingTime="2.25921355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:30:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:30:13.786614586 +0000 UTC m=+415.070665126" watchObservedRunningTime="2026-07-06 20:30:13.787800909 +0000 UTC m=+415.071851450"
Jul 06 20:30:30.520868 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.520831    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"]
Jul 06 20:30:30.527094 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.527072    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.543279 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.543256    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 06 20:30:30.543437 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.543319    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 06 20:30:30.543693 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.543680    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-l66h4\""
Jul 06 20:30:30.543957 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.543941    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 06 20:30:30.546178 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.546160    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 06 20:30:30.581004 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.580984    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"]
Jul 06 20:30:30.630349 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.630312    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.630488 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.630371    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-manifests\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.630488 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.630414    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-webhook-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.630488 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.630459    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjfkj\" (UniqueName: \"kubernetes.io/projected/eb185922-ee38-4b6b-8b0e-a66c89bf882e-kube-api-access-cjfkj\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.630663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.630489    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-tmp\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731079 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731036    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjfkj\" (UniqueName: \"kubernetes.io/projected/eb185922-ee38-4b6b-8b0e-a66c89bf882e-kube-api-access-cjfkj\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731079 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731077    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-tmp\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731343 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731101    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731343 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731130    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-manifests\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731343 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731162    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-webhook-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.731642 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.731616    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-manifests\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.733488 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.733466    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/eb185922-ee38-4b6b-8b0e-a66c89bf882e-tmp\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.733600 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.733582    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-webhook-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.733658 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.733616    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/eb185922-ee38-4b6b-8b0e-a66c89bf882e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.743565 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.743522    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjfkj\" (UniqueName: \"kubernetes.io/projected/eb185922-ee38-4b6b-8b0e-a66c89bf882e-kube-api-access-cjfkj\") pod \"opendatahub-operator-controller-manager-656f947554-9tkc8\" (UID: \"eb185922-ee38-4b6b-8b0e-a66c89bf882e\") " pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.836730 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.836647    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:30.984770 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:30.984736    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"]
Jul 06 20:30:30.997011 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:30:30.996984    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb185922_ee38_4b6b_8b0e_a66c89bf882e.slice/crio-5f437c7553e25d2ab02ad41d021f27e04442e8cd3940a29ef5d5dc5e635bfc08 WatchSource:0}: Error finding container 5f437c7553e25d2ab02ad41d021f27e04442e8cd3940a29ef5d5dc5e635bfc08: Status 404 returned error can't find the container with id 5f437c7553e25d2ab02ad41d021f27e04442e8cd3940a29ef5d5dc5e635bfc08
Jul 06 20:30:31.834059 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:31.834019    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8" event={"ID":"eb185922-ee38-4b6b-8b0e-a66c89bf882e","Type":"ContainerStarted","Data":"5f437c7553e25d2ab02ad41d021f27e04442e8cd3940a29ef5d5dc5e635bfc08"}
Jul 06 20:30:34.848462 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:34.848370    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8" event={"ID":"eb185922-ee38-4b6b-8b0e-a66c89bf882e","Type":"ContainerStarted","Data":"30393c96daf16180bb547691c371473cdadf34bf566e9a94e340b30cfe1653f1"}
Jul 06 20:30:35.853938 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:35.853897    2554 generic.go:356] "Generic (PLEG): container finished" podID="eb185922-ee38-4b6b-8b0e-a66c89bf882e" containerID="30393c96daf16180bb547691c371473cdadf34bf566e9a94e340b30cfe1653f1" exitCode=0
Jul 06 20:30:35.854334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:35.853945    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8" event={"ID":"eb185922-ee38-4b6b-8b0e-a66c89bf882e","Type":"ContainerDied","Data":"30393c96daf16180bb547691c371473cdadf34bf566e9a94e340b30cfe1653f1"}
Jul 06 20:30:36.859808 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:36.859769    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8" event={"ID":"eb185922-ee38-4b6b-8b0e-a66c89bf882e","Type":"ContainerStarted","Data":"87537781be3001198020c86f187a17e4539c4dbe8d3ff3b9ad7af6685c8177ee"}
Jul 06 20:30:36.860274 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:36.859849    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:36.913764 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:36.913717    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8" podStartSLOduration=2.015930834 podStartE2EDuration="6.913698868s" totalImagesPullingTime="3.828094342s" totalInitContainerRuntime="1.069673692s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:30:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:30:36.912616081 +0000 UTC m=+438.196666625" watchObservedRunningTime="2026-07-06 20:30:36.913698868 +0000 UTC m=+438.197749408"
Jul 06 20:30:47.867127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:47.867096    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-656f947554-9tkc8"
Jul 06 20:30:52.728667 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.728635    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:30:52.731683 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.731666    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.734085 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.734063    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 06 20:30:52.735311 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.735285    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bwmhq\""
Jul 06 20:30:52.735414 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.735335    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 06 20:30:52.736334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.736319    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 06 20:30:52.736663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.736643    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 06 20:30:52.736663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.736647    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 20:30:52.736889 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.736874    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 20:30:52.755807 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.755782    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:30:52.779881 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.779851    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.779886    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.779914    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vm89\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.779975    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.780017    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780149 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.780038    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.780149 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.780084    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881153 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881121    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881153 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881159    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881368 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881185    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881368 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881307    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vm89\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881368 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881361    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881504 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881414    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881504 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881439    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.881824 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.881795    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.883630 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.883604    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.883744 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.883675    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.883855 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.883836    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.883911 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.883883    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.913016 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.912983    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:52.914212 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:52.914192    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vm89\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89\") pod \"istiod-openshift-gateway-7574459656-tz9rj\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:53.040890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:53.040860    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:53.177819 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:53.177788    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:30:53.180656 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:30:53.180627    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0286ab3f_8e1d_4c0d_9551_dcc628f34009.slice/crio-cac87493d379d8bd20c49deb80742f82f5a41001a3226b440c9105fa777fe737 WatchSource:0}: Error finding container cac87493d379d8bd20c49deb80742f82f5a41001a3226b440c9105fa777fe737: Status 404 returned error can't find the container with id cac87493d379d8bd20c49deb80742f82f5a41001a3226b440c9105fa777fe737
Jul 06 20:30:53.922212 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:53.922177    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" event={"ID":"0286ab3f-8e1d-4c0d-9551-dcc628f34009","Type":"ContainerStarted","Data":"cac87493d379d8bd20c49deb80742f82f5a41001a3226b440c9105fa777fe737"}
Jul 06 20:30:55.359353 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:55.359312    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 06 20:30:55.359637 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:55.359405    2554 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
Jul 06 20:30:55.930569 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:55.930505    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" event={"ID":"0286ab3f-8e1d-4c0d-9551-dcc628f34009","Type":"ContainerStarted","Data":"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"}
Jul 06 20:30:55.930746 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:55.930654    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:30:55.954430 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:55.954368    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" podStartSLOduration=1.777913056 podStartE2EDuration="3.954339623s" totalImagesPullingTime="2.176426567s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:30:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:30:55.953734025 +0000 UTC m=+457.237784578" watchObservedRunningTime="2026-07-06 20:30:55.954339623 +0000 UTC m=+457.238390165"
Jul 06 20:30:56.937257 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:30:56.937229    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:31:01.002140 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.002112    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"]
Jul 06 20:31:01.005089 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.005073    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.013315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.013297    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 06 20:31:01.014332 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.014317    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-wbnmh\""
Jul 06 20:31:01.014540 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.014519    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 06 20:31:01.014585 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.014541    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 06 20:31:01.047233 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.047192    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.047399 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.047246    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbl7b\" (UniqueName: \"kubernetes.io/projected/1d79b6fc-4720-4b90-955c-1beb9e81faf9-kube-api-access-qbl7b\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.047399 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.047268    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-metrics-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.047399 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.047351    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d79b6fc-4720-4b90-955c-1beb9e81faf9-manager-config\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.108779 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.108745    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"]
Jul 06 20:31:01.147950 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.147910    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbl7b\" (UniqueName: \"kubernetes.io/projected/1d79b6fc-4720-4b90-955c-1beb9e81faf9-kube-api-access-qbl7b\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.147950 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.147951    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-metrics-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.148170 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.147990    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d79b6fc-4720-4b90-955c-1beb9e81faf9-manager-config\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.148170 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.148028    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.148707 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.148669    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/1d79b6fc-4720-4b90-955c-1beb9e81faf9-manager-config\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.150517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.150494    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-metrics-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.150637 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.150587    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d79b6fc-4720-4b90-955c-1beb9e81faf9-cert\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.169684 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.169654    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbl7b\" (UniqueName: \"kubernetes.io/projected/1d79b6fc-4720-4b90-955c-1beb9e81faf9-kube-api-access-qbl7b\") pod \"lws-controller-manager-6cf865c78f-jm5kg\" (UID: \"1d79b6fc-4720-4b90-955c-1beb9e81faf9\") " pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.313163 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.313127    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:01.450939 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.450905    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"]
Jul 06 20:31:01.453862 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:31:01.453834    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d79b6fc_4720_4b90_955c_1beb9e81faf9.slice/crio-3e93a9bab73f178da67f1740a99c2f28d3d7bed0991aca46783dec363cae39c1 WatchSource:0}: Error finding container 3e93a9bab73f178da67f1740a99c2f28d3d7bed0991aca46783dec363cae39c1: Status 404 returned error can't find the container with id 3e93a9bab73f178da67f1740a99c2f28d3d7bed0991aca46783dec363cae39c1
Jul 06 20:31:01.953808 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:01.953768    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg" event={"ID":"1d79b6fc-4720-4b90-955c-1beb9e81faf9","Type":"ContainerStarted","Data":"3e93a9bab73f178da67f1740a99c2f28d3d7bed0991aca46783dec363cae39c1"}
Jul 06 20:31:04.254192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.254155    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:31:04.254684 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.254355    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" podUID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" containerName="discovery" containerID="cri-o://34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0" gracePeriod=30
Jul 06 20:31:04.521405 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.521359    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:31:04.575488 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575451    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vm89\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575507    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575577    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575612    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575668    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575687    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575937 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575728    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert\") pod \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\" (UID: \"0286ab3f-8e1d-4c0d-9551-dcc628f34009\") "
Jul 06 20:31:04.575937 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.575921    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:31:04.578510 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.578480    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts" (OuterVolumeSpecName: "cacerts") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:31:04.578772 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.578703    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token" (OuterVolumeSpecName: "istio-token") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:31:04.578772 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.578744    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs" (OuterVolumeSpecName: "local-certs") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:31:04.578963 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.578942    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:31:04.579021 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.578962    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:31:04.579674 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.579647    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89" (OuterVolumeSpecName: "kube-api-access-5vm89") pod "0286ab3f-8e1d-4c0d-9551-dcc628f34009" (UID: "0286ab3f-8e1d-4c0d-9551-dcc628f34009"). InnerVolumeSpecName "kube-api-access-5vm89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:31:04.677076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677044    2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-dns-cert\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677073    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vm89\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-kube-api-access-5vm89\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677076 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677084    2554 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-csr-ca-configmap\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677093    2554 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/0286ab3f-8e1d-4c0d-9551-dcc628f34009-local-certs\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677104    2554 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-token\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677118    2554 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-istio-kubeconfig\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.677314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.677131    2554 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/0286ab3f-8e1d-4c0d-9551-dcc628f34009-cacerts\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:04.968387 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.968294    2554 generic.go:356] "Generic (PLEG): container finished" podID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" containerID="34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0" exitCode=0
Jul 06 20:31:04.968387 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.968342    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"
Jul 06 20:31:04.968387 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.968354    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" event={"ID":"0286ab3f-8e1d-4c0d-9551-dcc628f34009","Type":"ContainerDied","Data":"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"}
Jul 06 20:31:04.968673 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.968425    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj" event={"ID":"0286ab3f-8e1d-4c0d-9551-dcc628f34009","Type":"ContainerDied","Data":"cac87493d379d8bd20c49deb80742f82f5a41001a3226b440c9105fa777fe737"}
Jul 06 20:31:04.968673 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:04.968456    2554 scope.go:117] "RemoveContainer" containerID="34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"
Jul 06 20:31:05.012663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.012627    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:31:05.021040 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.021013    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-tz9rj"]
Jul 06 20:31:05.146516 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.146490    2554 scope.go:117] "RemoveContainer" containerID="34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"
Jul 06 20:31:05.146902 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:31:05.146880    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0\": container with ID starting with 34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0 not found: ID does not exist" containerID="34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"
Jul 06 20:31:05.146974 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.146918    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0"} err="failed to get container status \"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0\": rpc error: code = NotFound desc = could not find container \"34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0\": container with ID starting with 34918cd040a983f724f15139210c53424bd3ab26ebf84bbd1b4cd171b0a99dc0 not found: ID does not exist"
Jul 06 20:31:05.281818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.281789    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" path="/var/lib/kubelet/pods/0286ab3f-8e1d-4c0d-9551-dcc628f34009/volumes"
Jul 06 20:31:05.973794 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.973759    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg" event={"ID":"1d79b6fc-4720-4b90-955c-1beb9e81faf9","Type":"ContainerStarted","Data":"bcc0f1f3f658d715f5370d4f135386b463b977086990dc3bda589e2f24eae4b4"}
Jul 06 20:31:05.973995 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.973893    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:05.996117 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:05.996063    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg" podStartSLOduration=2.250167688 podStartE2EDuration="5.996047966s" totalImagesPullingTime="3.745880278s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:31:05.995989201 +0000 UTC m=+467.280039742" watchObservedRunningTime="2026-07-06 20:31:05.996047966 +0000 UTC m=+467.280098568"
Jul 06 20:31:16.979244 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:16.979210    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6cf865c78f-jm5kg"
Jul 06 20:31:51.483123 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.483085    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:51.483618 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.483339    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" containerName="discovery"
Jul 06 20:31:51.483618 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.483351    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" containerName="discovery"
Jul 06 20:31:51.483618 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.483400    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0286ab3f-8e1d-4c0d-9551-dcc628f34009" containerName="discovery"
Jul 06 20:31:51.485933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.485914    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:51.488829 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.488806    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2kv5k\""
Jul 06 20:31:51.490049 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.490023    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 06 20:31:51.490325 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.490306    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 06 20:31:51.507198 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.507171    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:51.622053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.622012    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wr7p\" (UniqueName: \"kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p\") pod \"kuadrant-operator-catalog-mmjmv\" (UID: \"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:51.722791 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.722748    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wr7p\" (UniqueName: \"kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p\") pod \"kuadrant-operator-catalog-mmjmv\" (UID: \"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:51.733452 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.733388    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wr7p\" (UniqueName: \"kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p\") pod \"kuadrant-operator-catalog-mmjmv\" (UID: \"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45\") " pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:51.796722 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.796683    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:51.796975 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.796955    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:51.935806 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:51.935775    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:51.938762 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:31:51.938726    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d6e2e99_2b43_44e9_a9b5_cfe67a193e45.slice/crio-8661a1282a6a8e5eaf36bd7add717f9e3a390b146c94267ca3a33f5ca8965c08 WatchSource:0}: Error finding container 8661a1282a6a8e5eaf36bd7add717f9e3a390b146c94267ca3a33f5ca8965c08: Status 404 returned error can't find the container with id 8661a1282a6a8e5eaf36bd7add717f9e3a390b146c94267ca3a33f5ca8965c08
Jul 06 20:31:52.019460 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.019429    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rvq5"]
Jul 06 20:31:52.023627 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.023609    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:31:52.068151 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.068122    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rvq5"]
Jul 06 20:31:52.126666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.126630    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzdbl\" (UniqueName: \"kubernetes.io/projected/eada3974-58f8-45a6-9e42-0f592dc00f50-kube-api-access-qzdbl\") pod \"kuadrant-operator-catalog-5rvq5\" (UID: \"eada3974-58f8-45a6-9e42-0f592dc00f50\") " pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:31:52.140581 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.140545    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv" event={"ID":"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45","Type":"ContainerStarted","Data":"8661a1282a6a8e5eaf36bd7add717f9e3a390b146c94267ca3a33f5ca8965c08"}
Jul 06 20:31:52.227758 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.227719    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzdbl\" (UniqueName: \"kubernetes.io/projected/eada3974-58f8-45a6-9e42-0f592dc00f50-kube-api-access-qzdbl\") pod \"kuadrant-operator-catalog-5rvq5\" (UID: \"eada3974-58f8-45a6-9e42-0f592dc00f50\") " pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:31:52.247431 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.247395    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzdbl\" (UniqueName: \"kubernetes.io/projected/eada3974-58f8-45a6-9e42-0f592dc00f50-kube-api-access-qzdbl\") pod \"kuadrant-operator-catalog-5rvq5\" (UID: \"eada3974-58f8-45a6-9e42-0f592dc00f50\") " pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:31:52.332434 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.332341    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:31:52.483942 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:52.483902    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-5rvq5"]
Jul 06 20:31:52.486777 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:31:52.486748    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeada3974_58f8_45a6_9e42_0f592dc00f50.slice/crio-f33df3ff93014e9c15e8a15d8d2825635cea458995d014b578bbd3dcf41f6621 WatchSource:0}: Error finding container f33df3ff93014e9c15e8a15d8d2825635cea458995d014b578bbd3dcf41f6621: Status 404 returned error can't find the container with id f33df3ff93014e9c15e8a15d8d2825635cea458995d014b578bbd3dcf41f6621
Jul 06 20:31:53.146689 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:53.146650    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5" event={"ID":"eada3974-58f8-45a6-9e42-0f592dc00f50","Type":"ContainerStarted","Data":"f33df3ff93014e9c15e8a15d8d2825635cea458995d014b578bbd3dcf41f6621"}
Jul 06 20:31:55.155501 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.155469    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5" event={"ID":"eada3974-58f8-45a6-9e42-0f592dc00f50","Type":"ContainerStarted","Data":"1cfc910e5fca714543c2cfed0d29cdc8e13391030eea1129d4e4f7e4944e00de"}
Jul 06 20:31:55.156866 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.156844    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv" event={"ID":"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45","Type":"ContainerStarted","Data":"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"}
Jul 06 20:31:55.156945 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.156928    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv" podUID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" containerName="registry-server" containerID="cri-o://20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753" gracePeriod=2
Jul 06 20:31:55.196116 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.196063    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5" podStartSLOduration=2.483514875 podStartE2EDuration="4.196042917s" totalImagesPullingTime="1.712528042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:31:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:31:55.195671845 +0000 UTC m=+516.479722400" watchObservedRunningTime="2026-07-06 20:31:55.196042917 +0000 UTC m=+516.480093456"
Jul 06 20:31:55.395615 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.395591    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:55.552296 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.552265    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2wr7p\" (UniqueName: \"kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p\") pod \"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45\" (UID: \"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45\") "
Jul 06 20:31:55.554519 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.554495    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p" (OuterVolumeSpecName: "kube-api-access-2wr7p") pod "5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" (UID: "5d6e2e99-2b43-44e9-a9b5-cfe67a193e45"). InnerVolumeSpecName "kube-api-access-2wr7p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:31:55.653279 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:55.653240    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2wr7p\" (UniqueName: \"kubernetes.io/projected/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45-kube-api-access-2wr7p\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:31:56.162273 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.162236    2554 generic.go:356] "Generic (PLEG): container finished" podID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" containerID="20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753" exitCode=0
Jul 06 20:31:56.162729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.162292    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv"
Jul 06 20:31:56.162729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.162323    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv" event={"ID":"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45","Type":"ContainerDied","Data":"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"}
Jul 06 20:31:56.162729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.162394    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-mmjmv" event={"ID":"5d6e2e99-2b43-44e9-a9b5-cfe67a193e45","Type":"ContainerDied","Data":"8661a1282a6a8e5eaf36bd7add717f9e3a390b146c94267ca3a33f5ca8965c08"}
Jul 06 20:31:56.162729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.162415    2554 scope.go:117] "RemoveContainer" containerID="20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"
Jul 06 20:31:56.170323 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.170305    2554 scope.go:117] "RemoveContainer" containerID="20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"
Jul 06 20:31:56.170583 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:31:56.170564    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753\": container with ID starting with 20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753 not found: ID does not exist" containerID="20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"
Jul 06 20:31:56.170640 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.170593    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753"} err="failed to get container status \"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753\": rpc error: code = NotFound desc = could not find container \"20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753\": container with ID starting with 20a00c9182ca3c3f4314e4045eeae43bb37ad82488032f19d0da7019aee3f753 not found: ID does not exist"
Jul 06 20:31:56.188979 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.188954    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:56.194743 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:56.194724    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-mmjmv"]
Jul 06 20:31:57.280705 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:31:57.280671    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" path="/var/lib/kubelet/pods/5d6e2e99-2b43-44e9-a9b5-cfe67a193e45/volumes"
Jul 06 20:32:02.333424 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:02.333388    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:32:02.333931 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:02.333478    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:32:02.355999 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:02.355966    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:32:03.211157 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:03.211130    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-5rvq5"
Jul 06 20:32:24.194220 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.194182    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"]
Jul 06 20:32:24.194642 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.194437    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" containerName="registry-server"
Jul 06 20:32:24.194642 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.194448    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" containerName="registry-server"
Jul 06 20:32:24.194642 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.194501    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5d6e2e99-2b43-44e9-a9b5-cfe67a193e45" containerName="registry-server"
Jul 06 20:32:24.198949 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.198931    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:24.202087 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.202059    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-wh8dr\""
Jul 06 20:32:24.202245 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.202228    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 06 20:32:24.208982 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.208962    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"]
Jul 06 20:32:24.271737 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.271703    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smmpz\" (UniqueName: \"kubernetes.io/projected/fddc4ecc-2c28-46b6-bee6-43dfbded20c4-kube-api-access-smmpz\") pod \"dns-operator-controller-manager-864dd78c4c-7gcjk\" (UID: \"fddc4ecc-2c28-46b6-bee6-43dfbded20c4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:24.372670 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.372637    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smmpz\" (UniqueName: \"kubernetes.io/projected/fddc4ecc-2c28-46b6-bee6-43dfbded20c4-kube-api-access-smmpz\") pod \"dns-operator-controller-manager-864dd78c4c-7gcjk\" (UID: \"fddc4ecc-2c28-46b6-bee6-43dfbded20c4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:24.406955 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.406918    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-smmpz\" (UniqueName: \"kubernetes.io/projected/fddc4ecc-2c28-46b6-bee6-43dfbded20c4-kube-api-access-smmpz\") pod \"dns-operator-controller-manager-864dd78c4c-7gcjk\" (UID: \"fddc4ecc-2c28-46b6-bee6-43dfbded20c4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:24.508952 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.508842    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:24.640997 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:24.640970    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"]
Jul 06 20:32:24.643796 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:32:24.643771    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfddc4ecc_2c28_46b6_bee6_43dfbded20c4.slice/crio-1c8537ab43b790fa50b6094cc96f0aae656487e1b56ce3854a42e0cc130d919f WatchSource:0}: Error finding container 1c8537ab43b790fa50b6094cc96f0aae656487e1b56ce3854a42e0cc130d919f: Status 404 returned error can't find the container with id 1c8537ab43b790fa50b6094cc96f0aae656487e1b56ce3854a42e0cc130d919f
Jul 06 20:32:25.280375 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:25.280339    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk" event={"ID":"fddc4ecc-2c28-46b6-bee6-43dfbded20c4","Type":"ContainerStarted","Data":"1c8537ab43b790fa50b6094cc96f0aae656487e1b56ce3854a42e0cc130d919f"}
Jul 06 20:32:27.284388 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:27.284352    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk" event={"ID":"fddc4ecc-2c28-46b6-bee6-43dfbded20c4","Type":"ContainerStarted","Data":"e40ad972af078995c30099abb056d7a9be6e17343b762579e638e0a2ffee8abd"}
Jul 06 20:32:27.284815 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:27.284482    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:27.307339 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:27.307289    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk" podStartSLOduration=1.654901881 podStartE2EDuration="3.30727425s" totalImagesPullingTime="1.652372369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:32:27.306770491 +0000 UTC m=+548.590821033" watchObservedRunningTime="2026-07-06 20:32:27.30727425 +0000 UTC m=+548.591324790"
Jul 06 20:32:29.531953 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.531917    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"]
Jul 06 20:32:29.538447 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.538424    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.544026 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.544001    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6qt66\""
Jul 06 20:32:29.582235 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.582203    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"]
Jul 06 20:32:29.605963 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.605925    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.606121 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.605977    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5cm7\" (UniqueName: \"kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.707107 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.707070    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.707107 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.707111    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5cm7\" (UniqueName: \"kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.707465 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.707445    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.730891 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.730861    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5cm7\" (UniqueName: \"kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7\") pod \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.847964 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.847884    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:29.985870 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:29.985840    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"]
Jul 06 20:32:29.989055 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:32:29.989027    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b6e9dc8_93c4_4b0b_9832_687ce749e22c.slice/crio-90e4443b8ad093821101fb323cc08544cd849967b9193cecdd0466c984db3ed2 WatchSource:0}: Error finding container 90e4443b8ad093821101fb323cc08544cd849967b9193cecdd0466c984db3ed2: Status 404 returned error can't find the container with id 90e4443b8ad093821101fb323cc08544cd849967b9193cecdd0466c984db3ed2
Jul 06 20:32:30.295671 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:30.295639    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d" event={"ID":"4b6e9dc8-93c4-4b0b-9832-687ce749e22c","Type":"ContainerStarted","Data":"90e4443b8ad093821101fb323cc08544cd849967b9193cecdd0466c984db3ed2"}
Jul 06 20:32:35.320146 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:35.320049    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d" event={"ID":"4b6e9dc8-93c4-4b0b-9832-687ce749e22c","Type":"ContainerStarted","Data":"1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf"}
Jul 06 20:32:35.320146 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:35.320132    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:37.757443 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.757384    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d" podStartSLOduration=3.717681488 podStartE2EDuration="8.757360628s" totalImagesPullingTime="5.03967914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:32:35.342461851 +0000 UTC m=+556.626512394" watchObservedRunningTime="2026-07-06 20:32:37.757360628 +0000 UTC m=+559.041411180"
Jul 06 20:32:37.758561 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.758544    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"]
Jul 06 20:32:37.761706 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.761691    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.764052 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.764026    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 06 20:32:37.764052 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.764047    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 06 20:32:37.764223 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.764072    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-c8bz4\""
Jul 06 20:32:37.772132 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.772111    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"]
Jul 06 20:32:37.869121 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.869092    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.869121 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.869124    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rpp8m\" (UniqueName: \"kubernetes.io/projected/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-kube-api-access-rpp8m\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.869471 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.869147    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.970464 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.970431    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.970659 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.970582    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.970659 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.970602    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rpp8m\" (UniqueName: \"kubernetes.io/projected/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-kube-api-access-rpp8m\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.970749 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:32:37.970723    2554 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 06 20:32:37.970813 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:32:37.970798    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert podName:20b63b5c-fbc4-405d-9ade-fcf1be8a4cda nodeName:}" failed. No retries permitted until 2026-07-06 20:32:38.470773413 +0000 UTC m=+559.754823945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-8whhz" (UID: "20b63b5c-fbc4-405d-9ade-fcf1be8a4cda") : secret "plugin-serving-cert" not found
Jul 06 20:32:37.971179 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.971162    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:37.978863 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:37.978843    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rpp8m\" (UniqueName: \"kubernetes.io/projected/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-kube-api-access-rpp8m\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:38.289957 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:38.289929    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-7gcjk"
Jul 06 20:32:38.474136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:38.474102    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:38.476499 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:38.476473    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/20b63b5c-fbc4-405d-9ade-fcf1be8a4cda-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-8whhz\" (UID: \"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:38.670361 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:38.670281    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"
Jul 06 20:32:38.804078 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:38.803921    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz"]
Jul 06 20:32:38.806722 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:32:38.806685    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod20b63b5c_fbc4_405d_9ade_fcf1be8a4cda.slice/crio-5442fa83da801ab41b86d8168353e067f0172d5b16b868b408e125cf9fb79a2a WatchSource:0}: Error finding container 5442fa83da801ab41b86d8168353e067f0172d5b16b868b408e125cf9fb79a2a: Status 404 returned error can't find the container with id 5442fa83da801ab41b86d8168353e067f0172d5b16b868b408e125cf9fb79a2a
Jul 06 20:32:39.337629 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:39.337579    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz" event={"ID":"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda","Type":"ContainerStarted","Data":"5442fa83da801ab41b86d8168353e067f0172d5b16b868b408e125cf9fb79a2a"}
Jul 06 20:32:46.326176 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:46.326140    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:48.305847 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.305802    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"]
Jul 06 20:32:48.306307 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.306052    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" containerName="manager" containerID="cri-o://1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf" gracePeriod=2
Jul 06 20:32:48.311851 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.311808    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"]
Jul 06 20:32:48.337112 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.337072    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:32:48.337469 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.337438    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" containerName="manager"
Jul 06 20:32:48.337469 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.337457    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" containerName="manager"
Jul 06 20:32:48.337663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.337543    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" containerName="manager"
Jul 06 20:32:48.348505 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.346251    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.365413 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.363938    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:32:48.387350 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:32:48.387293    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" is forbidden: User \"system:node:ip-10-0-128-47.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-47.ec2.internal' and this object" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:48.465509 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.465470    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pl98j\" (UniqueName: \"kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.465692 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.465620    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.566844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.566749    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pl98j\" (UniqueName: \"kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.567005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.566853    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.567286 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.567234    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.576544 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.576506    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pl98j\" (UniqueName: \"kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j\") pod \"kuadrant-operator-controller-manager-8684c9ff49-69smp\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:48.716850 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:48.716812    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:32:49.279626 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:32:49.279576    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" is forbidden: User \"system:node:ip-10-0-128-47.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-47.ec2.internal' and this object" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:49.586875 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.586796    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:32:49.618507 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.618477    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:32:49.618674 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.618605    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.678761 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.678724    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mxpk\" (UniqueName: \"kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.678911 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.678785    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.779741 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.779700    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mxpk\" (UniqueName: \"kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.779945 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.779776    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.780154 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.780133    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.790933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.790903    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mxpk\" (UniqueName: \"kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk\") pod \"kuadrant-operator-controller-manager-86878c769f-cpbxv\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:49.930498 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:49.930136    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:32:59.876902 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.876877    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:59.878771 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:32:59.878744    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" is forbidden: User \"system:node:ip-10-0-128-47.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-47.ec2.internal' and this object" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:32:59.885020 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.884995    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:32:59.888201 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:32:59.888179    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ad77db_b1cb_4289_9e7a_1f5092d536dd.slice/crio-bf06ff6a666e682add06bf9736ef6f74098d29780ee6a1a8840295eb47ec6834 WatchSource:0}: Error finding container bf06ff6a666e682add06bf9736ef6f74098d29780ee6a1a8840295eb47ec6834: Status 404 returned error can't find the container with id bf06ff6a666e682add06bf9736ef6f74098d29780ee6a1a8840295eb47ec6834
Jul 06 20:32:59.907009 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.906982    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:32:59.919548 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:32:59.919510    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7aadcb69_3ae8_43f2_aa9a_b9375dcb4fe5.slice/crio-0a769477ee6a690abae1eb1cf2ca71b0e3cc4285646c94b0fb1fc18ce7583d51 WatchSource:0}: Error finding container 0a769477ee6a690abae1eb1cf2ca71b0e3cc4285646c94b0fb1fc18ce7583d51: Status 404 returned error can't find the container with id 0a769477ee6a690abae1eb1cf2ca71b0e3cc4285646c94b0fb1fc18ce7583d51
Jul 06 20:32:59.970679 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.970661    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume\") pod \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") "
Jul 06 20:32:59.970776 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.970700    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5cm7\" (UniqueName: \"kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7\") pod \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\" (UID: \"4b6e9dc8-93c4-4b0b-9832-687ce749e22c\") "
Jul 06 20:32:59.971170 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.971148    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4b6e9dc8-93c4-4b0b-9832-687ce749e22c" (UID: "4b6e9dc8-93c4-4b0b-9832-687ce749e22c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:32:59.972439 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:32:59.972420    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7" (OuterVolumeSpecName: "kube-api-access-x5cm7") pod "4b6e9dc8-93c4-4b0b-9832-687ce749e22c" (UID: "4b6e9dc8-93c4-4b0b-9832-687ce749e22c"). InnerVolumeSpecName "kube-api-access-x5cm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:00.071864 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.071823    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-extensions-socket-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:00.071864 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.071856    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5cm7\" (UniqueName: \"kubernetes.io/projected/4b6e9dc8-93c4-4b0b-9832-687ce749e22c-kube-api-access-x5cm7\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:00.442411 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.442379    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz" event={"ID":"20b63b5c-fbc4-405d-9ade-fcf1be8a4cda","Type":"ContainerStarted","Data":"5ad08bc34d7e065347e8a2894010175f518a59fbab6cf0006eaead69393e6efe"}
Jul 06 20:33:00.443961 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.443934    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" event={"ID":"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5","Type":"ContainerStarted","Data":"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"}
Jul 06 20:33:00.444101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.443990    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:33:00.444101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.444013    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" event={"ID":"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5","Type":"ContainerStarted","Data":"0a769477ee6a690abae1eb1cf2ca71b0e3cc4285646c94b0fb1fc18ce7583d51"}
Jul 06 20:33:00.444660 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:00.444633    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" is forbidden: User \"system:node:ip-10-0-128-47.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-47.ec2.internal' and this object" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:33:00.445454 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.445436    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" event={"ID":"c2ad77db-b1cb-4289-9e7a-1f5092d536dd","Type":"ContainerStarted","Data":"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"}
Jul 06 20:33:00.445570 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.445485    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:33:00.445570 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.445511    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" event={"ID":"c2ad77db-b1cb-4289-9e7a-1f5092d536dd","Type":"ContainerStarted","Data":"bf06ff6a666e682add06bf9736ef6f74098d29780ee6a1a8840295eb47ec6834"}
Jul 06 20:33:00.446740 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.446710    2554 generic.go:356] "Generic (PLEG): container finished" podID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" containerID="1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf" exitCode=0
Jul 06 20:33:00.446853 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.446749    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:33:00.446853 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.446762    2554 scope.go:117] "RemoveContainer" containerID="1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf"
Jul 06 20:33:00.454944 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.454928    2554 scope.go:117] "RemoveContainer" containerID="1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf"
Jul 06 20:33:00.455174 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:00.455160    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf\": container with ID starting with 1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf not found: ID does not exist" containerID="1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf"
Jul 06 20:33:00.455213 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.455182    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf"} err="failed to get container status \"1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf\": rpc error: code = NotFound desc = could not find container \"1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf\": container with ID starting with 1610d02519c0b70f3c11d5b0a615132f2c378151082ff7aa6605f8346b50f0cf not found: ID does not exist"
Jul 06 20:33:00.468222 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.468182    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-8whhz" podStartSLOduration=2.465202487 podStartE2EDuration="23.468162956s" totalImagesPullingTime="21.002960469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:32:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:00.467417154 +0000 UTC m=+581.751467696" watchObservedRunningTime="2026-07-06 20:33:00.468162956 +0000 UTC m=+581.752213497"
Jul 06 20:33:00.493566 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.493493    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" podStartSLOduration=12.49347475 podStartE2EDuration="12.49347475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:32:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:00.492041775 +0000 UTC m=+581.776092318" watchObservedRunningTime="2026-07-06 20:33:00.49347475 +0000 UTC m=+581.777525291"
Jul 06 20:33:00.512312 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:00.512255    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" podStartSLOduration=11.51223286 podStartE2EDuration="11.51223286s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:00.511292428 +0000 UTC m=+581.795342969" watchObservedRunningTime="2026-07-06 20:33:00.51223286 +0000 UTC m=+581.796283402"
Jul 06 20:33:00.513345 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:00.513317    2554 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-cfx2d\" is forbidden: User \"system:node:ip-10-0-128-47.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-47.ec2.internal' and this object" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-cfx2d"
Jul 06 20:33:01.281310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:01.281274    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b6e9dc8-93c4-4b0b-9832-687ce749e22c" path="/var/lib/kubelet/pods/4b6e9dc8-93c4-4b0b-9832-687ce749e22c/volumes"
Jul 06 20:33:11.454067 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.454032    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:33:11.454557 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.454197    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:33:11.528365 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.528324    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:33:11.528630 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.528591    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" podUID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" containerName="manager" containerID="cri-o://2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b" gracePeriod=10
Jul 06 20:33:11.766225 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.766202    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:33:11.870193 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.870155    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pl98j\" (UniqueName: \"kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j\") pod \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") "
Jul 06 20:33:11.870356 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.870217    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume\") pod \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\" (UID: \"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5\") "
Jul 06 20:33:11.870670 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.870645    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" (UID: "7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:33:11.872239 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.872215    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j" (OuterVolumeSpecName: "kube-api-access-pl98j") pod "7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" (UID: "7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5"). InnerVolumeSpecName "kube-api-access-pl98j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:11.971517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.971481    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pl98j\" (UniqueName: \"kubernetes.io/projected/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-kube-api-access-pl98j\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:11.971517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:11.971510    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5-extensions-socket-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:12.499816 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.499786    2554 generic.go:356] "Generic (PLEG): container finished" podID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" containerID="2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b" exitCode=0
Jul 06 20:33:12.500205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.499835    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"
Jul 06 20:33:12.500205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.499857    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" event={"ID":"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5","Type":"ContainerDied","Data":"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"}
Jul 06 20:33:12.500205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.499902    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp" event={"ID":"7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5","Type":"ContainerDied","Data":"0a769477ee6a690abae1eb1cf2ca71b0e3cc4285646c94b0fb1fc18ce7583d51"}
Jul 06 20:33:12.500205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.499929    2554 scope.go:117] "RemoveContainer" containerID="2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"
Jul 06 20:33:12.508582 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.508559    2554 scope.go:117] "RemoveContainer" containerID="2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"
Jul 06 20:33:12.508822 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:12.508805    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b\": container with ID starting with 2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b not found: ID does not exist" containerID="2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"
Jul 06 20:33:12.508881 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.508831    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b"} err="failed to get container status \"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b\": rpc error: code = NotFound desc = could not find container \"2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b\": container with ID starting with 2d4c949ab01397d94322df03b70b69d49c77af7a5eb6a5c45f2b406dbf1c816b not found: ID does not exist"
Jul 06 20:33:12.564619 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.564589    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:33:12.571511 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:12.571492    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-69smp"]
Jul 06 20:33:13.280966 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:13.280933    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" path="/var/lib/kubelet/pods/7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5/volumes"
Jul 06 20:33:19.134944 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:19.134919    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:33:19.135387 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:19.134947    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:33:30.031956 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.031919    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:30.032418 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.032197    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" containerName="manager"
Jul 06 20:33:30.032418 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.032208    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" containerName="manager"
Jul 06 20:33:30.032418 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.032267    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aadcb69-3ae8-43f2-aa9a-b9375dcb4fe5" containerName="manager"
Jul 06 20:33:30.096509 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.096480    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:30.096703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.096628    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.099132 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.099111    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Jul 06 20:33:30.109782 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.109756    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x75rg\" (UniqueName: \"kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.109922 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.109802    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.136462 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.136434    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:30.211073 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.211039    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x75rg\" (UniqueName: \"kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.211073 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.211080    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.211742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.211723    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.220056 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.220023    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x75rg\" (UniqueName: \"kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg\") pod \"limitador-limitador-64c94fd64f-xfvwk\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.406848 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.406762    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:30.536695 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.536666    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:30.539742 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:33:30.539713    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f6006a9_01d6_4701_8ce8_b1da6756ee43.slice/crio-86711a9cb778207bc53e7bac9960d0e787979b5d38eea3c19b45de40af9d0b56 WatchSource:0}: Error finding container 86711a9cb778207bc53e7bac9960d0e787979b5d38eea3c19b45de40af9d0b56: Status 404 returned error can't find the container with id 86711a9cb778207bc53e7bac9960d0e787979b5d38eea3c19b45de40af9d0b56
Jul 06 20:33:30.571279 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.571249    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" event={"ID":"3f6006a9-01d6-4701-8ce8-b1da6756ee43","Type":"ContainerStarted","Data":"86711a9cb778207bc53e7bac9960d0e787979b5d38eea3c19b45de40af9d0b56"}
Jul 06 20:33:30.877812 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.877781    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:30.882226 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.882199    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:30.884716 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.884686    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-pwjkt\""
Jul 06 20:33:30.888767 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.888743    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:30.915156 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:30.915129    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbft\" (UniqueName: \"kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft\") pod \"authorino-58cd955675-hlcp2\" (UID: \"e28627c6-4368-4ae5-8817-8889fe920af6\") " pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:31.016204 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.016169    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djbft\" (UniqueName: \"kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft\") pod \"authorino-58cd955675-hlcp2\" (UID: \"e28627c6-4368-4ae5-8817-8889fe920af6\") " pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:31.026634 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.026601    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbft\" (UniqueName: \"kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft\") pod \"authorino-58cd955675-hlcp2\" (UID: \"e28627c6-4368-4ae5-8817-8889fe920af6\") " pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:31.095328 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.095291    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:33:31.098675 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.098636    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:33:31.106517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.106479    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:33:31.116953 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.116907    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h5jq\" (UniqueName: \"kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq\") pod \"authorino-75d556bd77-rq476\" (UID: \"7fd0d9cb-2870-4362-a129-bf58e3db2acc\") " pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:33:31.193840 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.193056    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:31.217827 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.217794    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h5jq\" (UniqueName: \"kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq\") pod \"authorino-75d556bd77-rq476\" (UID: \"7fd0d9cb-2870-4362-a129-bf58e3db2acc\") " pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:33:31.236544 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.236465    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h5jq\" (UniqueName: \"kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq\") pod \"authorino-75d556bd77-rq476\" (UID: \"7fd0d9cb-2870-4362-a129-bf58e3db2acc\") " pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:33:31.380995 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.379701    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:31.411491 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.411456    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:33:31.573602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.573516    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:33:31.578473 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:33:31.578439    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7fd0d9cb_2870_4362_a129_bf58e3db2acc.slice/crio-b0de3dbc4f23f1645f44bd0f783b3fb4d3c4a08ee4008a7a7100fa16b951e732 WatchSource:0}: Error finding container b0de3dbc4f23f1645f44bd0f783b3fb4d3c4a08ee4008a7a7100fa16b951e732: Status 404 returned error can't find the container with id b0de3dbc4f23f1645f44bd0f783b3fb4d3c4a08ee4008a7a7100fa16b951e732
Jul 06 20:33:31.582601 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:31.582558    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlcp2" event={"ID":"e28627c6-4368-4ae5-8817-8889fe920af6","Type":"ContainerStarted","Data":"d07c4c580821fadecbe2ad04453712382b3a6e050f512c1e76964fd6155f099d"}
Jul 06 20:33:32.592952 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:32.592909    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rq476" event={"ID":"7fd0d9cb-2870-4362-a129-bf58e3db2acc","Type":"ContainerStarted","Data":"b0de3dbc4f23f1645f44bd0f783b3fb4d3c4a08ee4008a7a7100fa16b951e732"}
Jul 06 20:33:35.610149 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.610110    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlcp2" event={"ID":"e28627c6-4368-4ae5-8817-8889fe920af6","Type":"ContainerStarted","Data":"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"}
Jul 06 20:33:35.611577 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.611551    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" event={"ID":"3f6006a9-01d6-4701-8ce8-b1da6756ee43","Type":"ContainerStarted","Data":"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"}
Jul 06 20:33:35.611678 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.611615    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:35.612917 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.612888    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rq476" event={"ID":"7fd0d9cb-2870-4362-a129-bf58e3db2acc","Type":"ContainerStarted","Data":"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"}
Jul 06 20:33:35.623585 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.623487    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-hlcp2" podStartSLOduration=1.567292879 podStartE2EDuration="5.62347101s" totalImagesPullingTime="4.056178131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:35.623103289 +0000 UTC m=+616.907153842" watchObservedRunningTime="2026-07-06 20:33:35.62347101 +0000 UTC m=+616.907521550"
Jul 06 20:33:35.638857 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.638810    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rq476" podStartSLOduration=0.765610485 podStartE2EDuration="4.638793642s" totalImagesPullingTime="3.873183157s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:35.637882192 +0000 UTC m=+616.921932734" watchObservedRunningTime="2026-07-06 20:33:35.638793642 +0000 UTC m=+616.922844184"
Jul 06 20:33:35.659735 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.659675    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" podStartSLOduration=0.82932056 podStartE2EDuration="5.65965406s" totalImagesPullingTime="4.8303335s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:35.659201892 +0000 UTC m=+616.943252433" watchObservedRunningTime="2026-07-06 20:33:35.65965406 +0000 UTC m=+616.943704602"
Jul 06 20:33:35.672113 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:35.672088    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:37.620487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:37.620448    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-hlcp2" podUID="e28627c6-4368-4ae5-8817-8889fe920af6" containerName="authorino" containerID="cri-o://44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a" gracePeriod=30
Jul 06 20:33:37.857061 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:37.857032    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:37.872446 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:37.872372    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djbft\" (UniqueName: \"kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft\") pod \"e28627c6-4368-4ae5-8817-8889fe920af6\" (UID: \"e28627c6-4368-4ae5-8817-8889fe920af6\") "
Jul 06 20:33:37.874599 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:37.874569    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft" (OuterVolumeSpecName: "kube-api-access-djbft") pod "e28627c6-4368-4ae5-8817-8889fe920af6" (UID: "e28627c6-4368-4ae5-8817-8889fe920af6"). InnerVolumeSpecName "kube-api-access-djbft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:37.973033 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:37.972998    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djbft\" (UniqueName: \"kubernetes.io/projected/e28627c6-4368-4ae5-8817-8889fe920af6-kube-api-access-djbft\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:38.626430 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.626397    2554 generic.go:356] "Generic (PLEG): container finished" podID="e28627c6-4368-4ae5-8817-8889fe920af6" containerID="44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a" exitCode=0
Jul 06 20:33:38.626934 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.626437    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-hlcp2"
Jul 06 20:33:38.626934 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.626441    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlcp2" event={"ID":"e28627c6-4368-4ae5-8817-8889fe920af6","Type":"ContainerDied","Data":"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"}
Jul 06 20:33:38.626934 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.626583    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-hlcp2" event={"ID":"e28627c6-4368-4ae5-8817-8889fe920af6","Type":"ContainerDied","Data":"d07c4c580821fadecbe2ad04453712382b3a6e050f512c1e76964fd6155f099d"}
Jul 06 20:33:38.626934 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.626611    2554 scope.go:117] "RemoveContainer" containerID="44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"
Jul 06 20:33:38.634447 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.634432    2554 scope.go:117] "RemoveContainer" containerID="44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"
Jul 06 20:33:38.634698 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:38.634680    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a\": container with ID starting with 44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a not found: ID does not exist" containerID="44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"
Jul 06 20:33:38.634767 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.634704    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a"} err="failed to get container status \"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a\": rpc error: code = NotFound desc = could not find container \"44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a\": container with ID starting with 44ab91293c8470873b97e0e2dd9706a5f27919b479d03d2cbbb7e0da3ae2712a not found: ID does not exist"
Jul 06 20:33:38.646135 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.646111    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:38.649663 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:38.649644    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-hlcp2"]
Jul 06 20:33:39.281901 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:39.281873    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e28627c6-4368-4ae5-8817-8889fe920af6" path="/var/lib/kubelet/pods/e28627c6-4368-4ae5-8817-8889fe920af6/volumes"
Jul 06 20:33:44.511461 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:44.511379    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:44.511847 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:44.511637    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" podUID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" containerName="limitador" containerID="cri-o://d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea" gracePeriod=30
Jul 06 20:33:44.512307 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:44.512246    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:45.054917 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.054894    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:45.127830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.127757    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x75rg\" (UniqueName: \"kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg\") pod \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") "
Jul 06 20:33:45.127830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.127802    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file\") pod \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\" (UID: \"3f6006a9-01d6-4701-8ce8-b1da6756ee43\") "
Jul 06 20:33:45.128201 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.128172    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file" (OuterVolumeSpecName: "config-file") pod "3f6006a9-01d6-4701-8ce8-b1da6756ee43" (UID: "3f6006a9-01d6-4701-8ce8-b1da6756ee43"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:33:45.129900 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.129871    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg" (OuterVolumeSpecName: "kube-api-access-x75rg") pod "3f6006a9-01d6-4701-8ce8-b1da6756ee43" (UID: "3f6006a9-01d6-4701-8ce8-b1da6756ee43"). InnerVolumeSpecName "kube-api-access-x75rg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:33:45.228412 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.228373    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x75rg\" (UniqueName: \"kubernetes.io/projected/3f6006a9-01d6-4701-8ce8-b1da6756ee43-kube-api-access-x75rg\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:45.228412 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.228408    2554 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f6006a9-01d6-4701-8ce8-b1da6756ee43-config-file\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:33:45.658330 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.658301    2554 generic.go:356] "Generic (PLEG): container finished" podID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" containerID="d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea" exitCode=0
Jul 06 20:33:45.658771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.658341    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"
Jul 06 20:33:45.658771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.658386    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" event={"ID":"3f6006a9-01d6-4701-8ce8-b1da6756ee43","Type":"ContainerDied","Data":"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"}
Jul 06 20:33:45.658771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.658430    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-xfvwk" event={"ID":"3f6006a9-01d6-4701-8ce8-b1da6756ee43","Type":"ContainerDied","Data":"86711a9cb778207bc53e7bac9960d0e787979b5d38eea3c19b45de40af9d0b56"}
Jul 06 20:33:45.658771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.658451    2554 scope.go:117] "RemoveContainer" containerID="d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"
Jul 06 20:33:45.666081 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.666068    2554 scope.go:117] "RemoveContainer" containerID="d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"
Jul 06 20:33:45.666307 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:33:45.666289    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea\": container with ID starting with d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea not found: ID does not exist" containerID="d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"
Jul 06 20:33:45.666372 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.666319    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea"} err="failed to get container status \"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea\": rpc error: code = NotFound desc = could not find container \"d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea\": container with ID starting with d06db491ab456f1efd48616e20ec1d6b4aa5d9997fad8e358e30ea646ac2f6ea not found: ID does not exist"
Jul 06 20:33:45.690925 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.690900    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:45.702672 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:45.702651    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-xfvwk"]
Jul 06 20:33:46.609155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609125    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-47zw5"]
Jul 06 20:33:46.609409 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609396    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" containerName="limitador"
Jul 06 20:33:46.609453 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609411    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" containerName="limitador"
Jul 06 20:33:46.609453 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609422    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e28627c6-4368-4ae5-8817-8889fe920af6" containerName="authorino"
Jul 06 20:33:46.609453 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609428    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="e28627c6-4368-4ae5-8817-8889fe920af6" containerName="authorino"
Jul 06 20:33:46.609602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609482    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="e28627c6-4368-4ae5-8817-8889fe920af6" containerName="authorino"
Jul 06 20:33:46.609602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.609489    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" containerName="limitador"
Jul 06 20:33:46.612106 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.612088    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.615337 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.615314    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-mpzdd\""
Jul 06 20:33:46.615711 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.615690    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
Jul 06 20:33:46.632911 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.632880    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-47zw5"]
Jul 06 20:33:46.637707 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.637683    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhjcm\" (UniqueName: \"kubernetes.io/projected/657366a3-b1da-455f-87d0-db9c2ebd279a-kube-api-access-nhjcm\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.637849 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.637734    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/657366a3-b1da-455f-87d0-db9c2ebd279a-data\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.738476 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.738435    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/657366a3-b1da-455f-87d0-db9c2ebd279a-data\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.738955 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.738500    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhjcm\" (UniqueName: \"kubernetes.io/projected/657366a3-b1da-455f-87d0-db9c2ebd279a-kube-api-access-nhjcm\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.738955 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.738809    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/657366a3-b1da-455f-87d0-db9c2ebd279a-data\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.747607 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.747584    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhjcm\" (UniqueName: \"kubernetes.io/projected/657366a3-b1da-455f-87d0-db9c2ebd279a-kube-api-access-nhjcm\") pod \"postgres-69fb4594cf-47zw5\" (UID: \"657366a3-b1da-455f-87d0-db9c2ebd279a\") " pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:46.922719 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:46.922634    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:47.053410 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:47.053380    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-47zw5"]
Jul 06 20:33:47.056696 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:33:47.056665    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod657366a3_b1da_455f_87d0_db9c2ebd279a.slice/crio-d4e09d6d753e6d847373b87bc115d9d7e27dff4e1ffefea525c17ca57efed83e WatchSource:0}: Error finding container d4e09d6d753e6d847373b87bc115d9d7e27dff4e1ffefea525c17ca57efed83e: Status 404 returned error can't find the container with id d4e09d6d753e6d847373b87bc115d9d7e27dff4e1ffefea525c17ca57efed83e
Jul 06 20:33:47.280874 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:47.280848    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f6006a9-01d6-4701-8ce8-b1da6756ee43" path="/var/lib/kubelet/pods/3f6006a9-01d6-4701-8ce8-b1da6756ee43/volumes"
Jul 06 20:33:47.669126 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:47.669041    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-47zw5" event={"ID":"657366a3-b1da-455f-87d0-db9c2ebd279a","Type":"ContainerStarted","Data":"d4e09d6d753e6d847373b87bc115d9d7e27dff4e1ffefea525c17ca57efed83e"}
Jul 06 20:33:52.697349 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:52.697314    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-47zw5" event={"ID":"657366a3-b1da-455f-87d0-db9c2ebd279a","Type":"ContainerStarted","Data":"f3e0047c240a6341fb82aa6fdbbdf5762ed9ba79c9f53afc41008fa6c3e41875"}
Jul 06 20:33:52.697766 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:52.697453    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:52.718143 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:52.718100    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-47zw5" podStartSLOduration=1.861949059 podStartE2EDuration="6.718085667s" totalImagesPullingTime="4.856136608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:33:52.71703288 +0000 UTC m=+634.001083425" watchObservedRunningTime="2026-07-06 20:33:52.718085667 +0000 UTC m=+634.002136208"
Jul 06 20:33:58.730082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:58.730053    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-47zw5"
Jul 06 20:33:59.312454 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.312420    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:33:59.320717 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.320692    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:33:59.330274 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.330249    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:33:59.445819 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.445791    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-76z46\" (UniqueName: \"kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46\") pod \"authorino-5d4b67c446-4db7f\" (UID: \"ef8fb216-e299-4750-9e62-a23818677579\") " pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:33:59.546353 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.546317    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-76z46\" (UniqueName: \"kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46\") pod \"authorino-5d4b67c446-4db7f\" (UID: \"ef8fb216-e299-4750-9e62-a23818677579\") " pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:33:59.557243 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.557211    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-76z46\" (UniqueName: \"kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46\") pod \"authorino-5d4b67c446-4db7f\" (UID: \"ef8fb216-e299-4750-9e62-a23818677579\") " pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:33:59.574977 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.574911    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:33:59.575133 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.575121    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:33:59.719988 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:33:59.719959    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:33:59.722878 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:33:59.722841    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podef8fb216_e299_4750_9e62_a23818677579.slice/crio-469c630fb0c40a7d2eb90a8edfb75e31da38a1f82fb1462f3749b0df6e65c8ec WatchSource:0}: Error finding container 469c630fb0c40a7d2eb90a8edfb75e31da38a1f82fb1462f3749b0df6e65c8ec: Status 404 returned error can't find the container with id 469c630fb0c40a7d2eb90a8edfb75e31da38a1f82fb1462f3749b0df6e65c8ec
Jul 06 20:34:00.731371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:00.731283    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4db7f" event={"ID":"ef8fb216-e299-4750-9e62-a23818677579","Type":"ContainerStarted","Data":"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"}
Jul 06 20:34:00.731371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:00.731309    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-5d4b67c446-4db7f" podUID="ef8fb216-e299-4750-9e62-a23818677579" containerName="authorino" containerID="cri-o://2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e" gracePeriod=30
Jul 06 20:34:00.731371 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:00.731330    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4db7f" event={"ID":"ef8fb216-e299-4750-9e62-a23818677579","Type":"ContainerStarted","Data":"469c630fb0c40a7d2eb90a8edfb75e31da38a1f82fb1462f3749b0df6e65c8ec"}
Jul 06 20:34:00.749268 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:00.749214    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-5d4b67c446-4db7f" podStartSLOduration=1.290984623 podStartE2EDuration="1.749197324s" totalImagesPullingTime="458.212701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:33:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:00.747783078 +0000 UTC m=+642.031833620" watchObservedRunningTime="2026-07-06 20:34:00.749197324 +0000 UTC m=+642.033247864"
Jul 06 20:34:01.012093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.012067    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:34:01.159694 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.159664    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-76z46\" (UniqueName: \"kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46\") pod \"ef8fb216-e299-4750-9e62-a23818677579\" (UID: \"ef8fb216-e299-4750-9e62-a23818677579\") "
Jul 06 20:34:01.161660 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.161632    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46" (OuterVolumeSpecName: "kube-api-access-76z46") pod "ef8fb216-e299-4750-9e62-a23818677579" (UID: "ef8fb216-e299-4750-9e62-a23818677579"). InnerVolumeSpecName "kube-api-access-76z46". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:34:01.261172 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.261143    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-76z46\" (UniqueName: \"kubernetes.io/projected/ef8fb216-e299-4750-9e62-a23818677579-kube-api-access-76z46\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:34:01.737079 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.736994    2554 generic.go:356] "Generic (PLEG): container finished" podID="ef8fb216-e299-4750-9e62-a23818677579" containerID="2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e" exitCode=0
Jul 06 20:34:01.737079 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.737044    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-4db7f"
Jul 06 20:34:01.737591 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.737081    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4db7f" event={"ID":"ef8fb216-e299-4750-9e62-a23818677579","Type":"ContainerDied","Data":"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"}
Jul 06 20:34:01.737591 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.737126    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-5d4b67c446-4db7f" event={"ID":"ef8fb216-e299-4750-9e62-a23818677579","Type":"ContainerDied","Data":"469c630fb0c40a7d2eb90a8edfb75e31da38a1f82fb1462f3749b0df6e65c8ec"}
Jul 06 20:34:01.737591 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.737146    2554 scope.go:117] "RemoveContainer" containerID="2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"
Jul 06 20:34:01.747797 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.747777    2554 scope.go:117] "RemoveContainer" containerID="2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"
Jul 06 20:34:01.748047 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:34:01.748030    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e\": container with ID starting with 2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e not found: ID does not exist" containerID="2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"
Jul 06 20:34:01.748102 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.748059    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e"} err="failed to get container status \"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e\": rpc error: code = NotFound desc = could not find container \"2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e\": container with ID starting with 2658ee41e597a1b884a8372aa9b868875d179c0298e4303c3ba254df836f197e not found: ID does not exist"
Jul 06 20:34:01.752183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.752156    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:34:01.756220 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:01.756191    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-4db7f"]
Jul 06 20:34:03.281224 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:03.281193    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef8fb216-e299-4750-9e62-a23818677579" path="/var/lib/kubelet/pods/ef8fb216-e299-4750-9e62-a23818677579/volumes"
Jul 06 20:34:03.676992 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:03.676913    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:34:03.677130 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:03.677077    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rq476" podUID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" containerName="authorino" containerID="cri-o://d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739" gracePeriod=30
Jul 06 20:34:04.456921 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.456896    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:34:04.590143 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.590056    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7h5jq\" (UniqueName: \"kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq\") pod \"7fd0d9cb-2870-4362-a129-bf58e3db2acc\" (UID: \"7fd0d9cb-2870-4362-a129-bf58e3db2acc\") "
Jul 06 20:34:04.592055 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.592033    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq" (OuterVolumeSpecName: "kube-api-access-7h5jq") pod "7fd0d9cb-2870-4362-a129-bf58e3db2acc" (UID: "7fd0d9cb-2870-4362-a129-bf58e3db2acc"). InnerVolumeSpecName "kube-api-access-7h5jq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:34:04.691350 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.691310    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7h5jq\" (UniqueName: \"kubernetes.io/projected/7fd0d9cb-2870-4362-a129-bf58e3db2acc-kube-api-access-7h5jq\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:34:04.751698 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.751661    2554 generic.go:356] "Generic (PLEG): container finished" podID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" containerID="d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739" exitCode=0
Jul 06 20:34:04.751862 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.751705    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rq476"
Jul 06 20:34:04.751862 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.751753    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rq476" event={"ID":"7fd0d9cb-2870-4362-a129-bf58e3db2acc","Type":"ContainerDied","Data":"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"}
Jul 06 20:34:04.751862 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.751798    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rq476" event={"ID":"7fd0d9cb-2870-4362-a129-bf58e3db2acc","Type":"ContainerDied","Data":"b0de3dbc4f23f1645f44bd0f783b3fb4d3c4a08ee4008a7a7100fa16b951e732"}
Jul 06 20:34:04.751862 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.751821    2554 scope.go:117] "RemoveContainer" containerID="d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"
Jul 06 20:34:04.763175 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.763147    2554 scope.go:117] "RemoveContainer" containerID="d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"
Jul 06 20:34:04.763520 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:34:04.763496    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739\": container with ID starting with d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739 not found: ID does not exist" containerID="d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"
Jul 06 20:34:04.763611 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.763557    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739"} err="failed to get container status \"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739\": rpc error: code = NotFound desc = could not find container \"d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739\": container with ID starting with d7a1635848292c91b2c173445a91f8459011d29050dd399993531074100c8739 not found: ID does not exist"
Jul 06 20:34:04.782424 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.782394    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:34:04.793683 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:04.793660    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rq476"]
Jul 06 20:34:05.280964 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:05.280937    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" path="/var/lib/kubelet/pods/7fd0d9cb-2870-4362-a129-bf58e3db2acc/volumes"
Jul 06 20:34:08.385553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385498    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385794    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" containerName="authorino"
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385805    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" containerName="authorino"
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385819    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef8fb216-e299-4750-9e62-a23818677579" containerName="authorino"
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385825    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef8fb216-e299-4750-9e62-a23818677579" containerName="authorino"
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385877    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef8fb216-e299-4750-9e62-a23818677579" containerName="authorino"
Jul 06 20:34:08.385933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.385887    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fd0d9cb-2870-4362-a129-bf58e3db2acc" containerName="authorino"
Jul 06 20:34:08.388344 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.388327    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.390890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.390867    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 20:34:08.391018 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.390981    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2qjmv\""
Jul 06 20:34:08.391018 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.391003    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 20:34:08.402927 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.402903    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:08.520877 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.520848    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.521037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.520925    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrx2k\" (UniqueName: \"kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.600511 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.600474    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:34:08.602962 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.602946    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.614952 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.614926    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:34:08.621731 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.621704    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrx2k\" (UniqueName: \"kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.621877 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.621742    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.624168 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.624139    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.635845 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.635779    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrx2k\" (UniqueName: \"kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k\") pod \"maas-controller-998456b46-btj57\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") " pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.698581 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.698524    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:08.722858 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.722821    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2gtr\" (UniqueName: \"kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.723032 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.722867    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.824211 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.824174    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2gtr\" (UniqueName: \"kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.824211 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.824216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.830680 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.830624    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.855968 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.838013    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:08.858666 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.858640    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2gtr\" (UniqueName: \"kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr\") pod \"maas-controller-846c44d545-f5ztx\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") " pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:08.913551 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:08.913450    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:09.044787 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:09.044751    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:34:09.047579 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:34:09.047551    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbf66be7_d892_4698_a22c_da62968941c9.slice/crio-5ccd8e24fc4c884356bde18f4c0ee44009e4fea50581572db28de8368bb603d0 WatchSource:0}: Error finding container 5ccd8e24fc4c884356bde18f4c0ee44009e4fea50581572db28de8368bb603d0: Status 404 returned error can't find the container with id 5ccd8e24fc4c884356bde18f4c0ee44009e4fea50581572db28de8368bb603d0
Jul 06 20:34:09.778771 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:09.778729    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-f5ztx" event={"ID":"bbf66be7-d892-4698-a22c-da62968941c9","Type":"ContainerStarted","Data":"5ccd8e24fc4c884356bde18f4c0ee44009e4fea50581572db28de8368bb603d0"}
Jul 06 20:34:09.780369 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:09.780338    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-998456b46-btj57" event={"ID":"768819b3-ffb0-4790-9eef-c7b85adc748b","Type":"ContainerStarted","Data":"a5ae749e5788f6a545c933279b7941619b78024a6f16f6bedfcd239011723e1d"}
Jul 06 20:34:11.790429 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.790391    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-f5ztx" event={"ID":"bbf66be7-d892-4698-a22c-da62968941c9","Type":"ContainerStarted","Data":"f9b3b192245b66b459ce557fe582e1a760168b70e916db60cdd07ce8663324ff"}
Jul 06 20:34:11.790922 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.790555    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:11.791840 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.791815    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-998456b46-btj57" event={"ID":"768819b3-ffb0-4790-9eef-c7b85adc748b","Type":"ContainerStarted","Data":"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"}
Jul 06 20:34:11.791962 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.791946    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:11.810767 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.810726    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-846c44d545-f5ztx" podStartSLOduration=1.911405048 podStartE2EDuration="3.810712345s" totalImagesPullingTime="1.899307297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:11.80980096 +0000 UTC m=+653.093851513" watchObservedRunningTime="2026-07-06 20:34:11.810712345 +0000 UTC m=+653.094762886"
Jul 06 20:34:11.828083 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:11.828038    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-998456b46-btj57" podStartSLOduration=1.745011536 podStartE2EDuration="3.828022237s" totalImagesPullingTime="2.083010701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:11.826879817 +0000 UTC m=+653.110930371" watchObservedRunningTime="2026-07-06 20:34:11.828022237 +0000 UTC m=+653.112072852"
Jul 06 20:34:22.801587 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:22.801519    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:22.802090 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:22.801616    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:34:22.859484 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:22.859453    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:22.859727 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:22.859676    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-998456b46-btj57" podUID="768819b3-ffb0-4790-9eef-c7b85adc748b" containerName="manager" containerID="cri-o://a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6" gracePeriod=30
Jul 06 20:34:23.101965 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.101944    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:23.242101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.242068    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert\") pod \"768819b3-ffb0-4790-9eef-c7b85adc748b\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") "
Jul 06 20:34:23.242101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.242106    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrx2k\" (UniqueName: \"kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k\") pod \"768819b3-ffb0-4790-9eef-c7b85adc748b\" (UID: \"768819b3-ffb0-4790-9eef-c7b85adc748b\") "
Jul 06 20:34:23.244330 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.244304    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "768819b3-ffb0-4790-9eef-c7b85adc748b" (UID: "768819b3-ffb0-4790-9eef-c7b85adc748b"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:34:23.244428 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.244322    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k" (OuterVolumeSpecName: "kube-api-access-nrx2k") pod "768819b3-ffb0-4790-9eef-c7b85adc748b" (UID: "768819b3-ffb0-4790-9eef-c7b85adc748b"). InnerVolumeSpecName "kube-api-access-nrx2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:34:23.343441 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.343354    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/768819b3-ffb0-4790-9eef-c7b85adc748b-webhook-cert\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:34:23.343441 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.343384    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrx2k\" (UniqueName: \"kubernetes.io/projected/768819b3-ffb0-4790-9eef-c7b85adc748b-kube-api-access-nrx2k\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:34:23.850991 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.850953    2554 generic.go:356] "Generic (PLEG): container finished" podID="768819b3-ffb0-4790-9eef-c7b85adc748b" containerID="a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6" exitCode=0
Jul 06 20:34:23.851425 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.851004    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-998456b46-btj57"
Jul 06 20:34:23.851425 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.851032    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-998456b46-btj57" event={"ID":"768819b3-ffb0-4790-9eef-c7b85adc748b","Type":"ContainerDied","Data":"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"}
Jul 06 20:34:23.851425 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.851087    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-998456b46-btj57" event={"ID":"768819b3-ffb0-4790-9eef-c7b85adc748b","Type":"ContainerDied","Data":"a5ae749e5788f6a545c933279b7941619b78024a6f16f6bedfcd239011723e1d"}
Jul 06 20:34:23.851425 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.851108    2554 scope.go:117] "RemoveContainer" containerID="a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"
Jul 06 20:34:23.859270 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.859253    2554 scope.go:117] "RemoveContainer" containerID="a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"
Jul 06 20:34:23.859516 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:34:23.859497    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6\": container with ID starting with a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6 not found: ID does not exist" containerID="a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"
Jul 06 20:34:23.859583 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.859547    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6"} err="failed to get container status \"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6\": rpc error: code = NotFound desc = could not find container \"a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6\": container with ID starting with a2d7907d55f211d3da725dce40872ed8cf67c8cdfdc4d1486071511d427ce8b6 not found: ID does not exist"
Jul 06 20:34:23.866952 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.866932    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:23.871515 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:23.871495    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-998456b46-btj57"]
Jul 06 20:34:25.281176 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:25.281143    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="768819b3-ffb0-4790-9eef-c7b85adc748b" path="/var/lib/kubelet/pods/768819b3-ffb0-4790-9eef-c7b85adc748b/volumes"
Jul 06 20:34:44.687625 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.687592    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7ff5fcbbd4-lct6m"]
Jul 06 20:34:44.688037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.687908    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="768819b3-ffb0-4790-9eef-c7b85adc748b" containerName="manager"
Jul 06 20:34:44.688037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.687920    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="768819b3-ffb0-4790-9eef-c7b85adc748b" containerName="manager"
Jul 06 20:34:44.688037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.687984    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="768819b3-ffb0-4790-9eef-c7b85adc748b" containerName="manager"
Jul 06 20:34:44.690729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.690712    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.693986 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.693965    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-jzfrq\""
Jul 06 20:34:44.695651 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.695633    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
Jul 06 20:34:44.707507 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.707481    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7ff5fcbbd4-lct6m"]
Jul 06 20:34:44.711120 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.711099    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985f9f65-be68-4028-9352-b326d5f2ebe7-openshift-service-ca\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.711217 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.711151    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xskn\" (UniqueName: \"kubernetes.io/projected/985f9f65-be68-4028-9352-b326d5f2ebe7-kube-api-access-6xskn\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.711217 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.711177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.811969 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.811938    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985f9f65-be68-4028-9352-b326d5f2ebe7-openshift-service-ca\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.812136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.811999    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xskn\" (UniqueName: \"kubernetes.io/projected/985f9f65-be68-4028-9352-b326d5f2ebe7-kube-api-access-6xskn\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.812136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.812026    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.812136 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:34:44.812117    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Jul 06 20:34:44.812305 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:34:44.812181    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls podName:985f9f65-be68-4028-9352-b326d5f2ebe7 nodeName:}" failed. No retries permitted until 2026-07-06 20:34:45.312164767 +0000 UTC m=+686.596215287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls") pod "maas-api-7ff5fcbbd4-lct6m" (UID: "985f9f65-be68-4028-9352-b326d5f2ebe7") : secret "maas-api-serving-cert" not found
Jul 06 20:34:44.812602 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.812583    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/985f9f65-be68-4028-9352-b326d5f2ebe7-openshift-service-ca\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:44.820811 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:44.820787    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xskn\" (UniqueName: \"kubernetes.io/projected/985f9f65-be68-4028-9352-b326d5f2ebe7-kube-api-access-6xskn\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:45.182517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.182476    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7rdmj"]
Jul 06 20:34:45.185890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.185871    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.188762 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.188732    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-xwjjq\""
Jul 06 20:34:45.188890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.188773    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 06 20:34:45.188890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.188784    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 06 20:34:45.188890 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.188732    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 06 20:34:45.208190 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.208161    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7rdmj"]
Jul 06 20:34:45.214894 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.214867    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.215006 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.214919    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rs5qg\" (UniqueName: \"kubernetes.io/projected/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-kube-api-access-rs5qg\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.228583 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.228547    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bksg8"]
Jul 06 20:34:45.231683 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.231666    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.244452 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.244428    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bksg8"]
Jul 06 20:34:45.316063 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316021    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:45.316260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316081    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.316260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316118    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/950c1428-0dca-4f1d-8ab6-539a0f22742e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.316260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316138    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rs5qg\" (UniqueName: \"kubernetes.io/projected/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-kube-api-access-rs5qg\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.316260 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316169    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxrn\" (UniqueName: \"kubernetes.io/projected/950c1428-0dca-4f1d-8ab6-539a0f22742e-kube-api-access-4nxrn\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.316864 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.316841    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-plugins-config-volume\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.318361 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.318340    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/985f9f65-be68-4028-9352-b326d5f2ebe7-maas-api-tls\") pod \"maas-api-7ff5fcbbd4-lct6m\" (UID: \"985f9f65-be68-4028-9352-b326d5f2ebe7\") " pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:45.324355 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.324330    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rs5qg\" (UniqueName: \"kubernetes.io/projected/da3d9586-cf9a-4202-844d-ae8bd56a5ae3-kube-api-access-rs5qg\") pod \"payload-pre-processing-7964457696-7rdmj\" (UID: \"da3d9586-cf9a-4202-844d-ae8bd56a5ae3\") " pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.417551 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.417492    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxrn\" (UniqueName: \"kubernetes.io/projected/950c1428-0dca-4f1d-8ab6-539a0f22742e-kube-api-access-4nxrn\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.417721 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.417616    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/950c1428-0dca-4f1d-8ab6-539a0f22742e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.418299 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.418279    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/950c1428-0dca-4f1d-8ab6-539a0f22742e-plugins-config-volume\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.426018 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.425995    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxrn\" (UniqueName: \"kubernetes.io/projected/950c1428-0dca-4f1d-8ab6-539a0f22742e-kube-api-access-4nxrn\") pod \"payload-processing-58f86c7798-bksg8\" (UID: \"950c1428-0dca-4f1d-8ab6-539a0f22742e\") " pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.496954 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.496876    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:45.541731 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.541692    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:45.600772 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.600745    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:45.645900 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.645771    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-7rdmj"]
Jul 06 20:34:45.710320 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.710294    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-bksg8"]
Jul 06 20:34:45.712431 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:34:45.712405    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod950c1428_0dca_4f1d_8ab6_539a0f22742e.slice/crio-2a20218d5288306d6bc6c8a6696ddf6d75a46168f9f95108a110b320f7e3b375 WatchSource:0}: Error finding container 2a20218d5288306d6bc6c8a6696ddf6d75a46168f9f95108a110b320f7e3b375: Status 404 returned error can't find the container with id 2a20218d5288306d6bc6c8a6696ddf6d75a46168f9f95108a110b320f7e3b375
Jul 06 20:34:45.759444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.759414    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7ff5fcbbd4-lct6m"]
Jul 06 20:34:45.761891 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:34:45.761864    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod985f9f65_be68_4028_9352_b326d5f2ebe7.slice/crio-3316045b1c1db57265fc06f02dff9f66db1d8a23bce4ed6ff9af9b917f1e48e6 WatchSource:0}: Error finding container 3316045b1c1db57265fc06f02dff9f66db1d8a23bce4ed6ff9af9b917f1e48e6: Status 404 returned error can't find the container with id 3316045b1c1db57265fc06f02dff9f66db1d8a23bce4ed6ff9af9b917f1e48e6
Jul 06 20:34:45.943768 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.943730    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj" event={"ID":"da3d9586-cf9a-4202-844d-ae8bd56a5ae3","Type":"ContainerStarted","Data":"cd3353f7124f4a8eaa8903245274a855f9e7914b2bc5a9fa770338d95ce64f5d"}
Jul 06 20:34:45.944884 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.944859    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m" event={"ID":"985f9f65-be68-4028-9352-b326d5f2ebe7","Type":"ContainerStarted","Data":"3316045b1c1db57265fc06f02dff9f66db1d8a23bce4ed6ff9af9b917f1e48e6"}
Jul 06 20:34:45.945941 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:45.945918    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bksg8" event={"ID":"950c1428-0dca-4f1d-8ab6-539a0f22742e","Type":"ContainerStarted","Data":"2a20218d5288306d6bc6c8a6696ddf6d75a46168f9f95108a110b320f7e3b375"}
Jul 06 20:34:46.953188 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.953149    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-bksg8" event={"ID":"950c1428-0dca-4f1d-8ab6-539a0f22742e","Type":"ContainerStarted","Data":"b639195b2fda6b2d1866f9cc26fe750267c5dad419b68342a1ad100d80d763f4"}
Jul 06 20:34:46.953598 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.953274    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:34:46.954926 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.954902    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj" event={"ID":"da3d9586-cf9a-4202-844d-ae8bd56a5ae3","Type":"ContainerStarted","Data":"45f71675d37f582a52d9e0505728bf5469e4fa9855d10ae6a69cbd87832fdb44"}
Jul 06 20:34:46.955061 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.954966    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:46.970703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.970657    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-bksg8" podStartSLOduration=0.82351748 podStartE2EDuration="1.970639746s" totalImagesPullingTime="1.147122266s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:46.969412665 +0000 UTC m=+688.253463203" watchObservedRunningTime="2026-07-06 20:34:46.970639746 +0000 UTC m=+688.254690286"
Jul 06 20:34:46.986523 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:46.986475    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj" podStartSLOduration=0.756001207 podStartE2EDuration="1.986455093s" totalImagesPullingTime="1.230453886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:46.98551174 +0000 UTC m=+688.269562283" watchObservedRunningTime="2026-07-06 20:34:46.986455093 +0000 UTC m=+688.270505636"
Jul 06 20:34:47.960640 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:47.960606    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m" event={"ID":"985f9f65-be68-4028-9352-b326d5f2ebe7","Type":"ContainerStarted","Data":"c4d02e296de93757f714d1ec5832e5db6f11c2e6b40a022f41e554c50aa69b20"}
Jul 06 20:34:47.961089 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:47.960981    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:47.978624 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:47.978579    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m" podStartSLOduration=2.384906271 podStartE2EDuration="3.978563928s" totalImagesPullingTime="1.593657657s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:34:47.978200658 +0000 UTC m=+689.262251197" watchObservedRunningTime="2026-07-06 20:34:47.978563928 +0000 UTC m=+689.262614469"
Jul 06 20:34:53.971369 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:53.971338    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-7ff5fcbbd4-lct6m"
Jul 06 20:34:57.962515 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:57.962484    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-7rdmj"
Jul 06 20:34:57.962968 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:34:57.962664    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-bksg8"
Jul 06 20:35:01.034346 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.034312    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"]
Jul 06 20:35:01.037783 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.037764    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.040195 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.040171    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 06 20:35:01.040308 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.040170    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6dg98\""
Jul 06 20:35:01.040840 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.040826    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 06 20:35:01.040919 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.040906    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 06 20:35:01.049544 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.049509    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"]
Jul 06 20:35:01.149237 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149201    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.149237 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149238    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.149487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149266    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.149487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149347    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.149487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149377    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.149487 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.149418    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqzkr\" (UniqueName: \"kubernetes.io/projected/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-kube-api-access-jqzkr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250100    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqzkr\" (UniqueName: \"kubernetes.io/projected/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-kube-api-access-jqzkr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250136 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250139    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250159    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250177    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250216    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250235    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250636 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250572    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250636 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250605    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.250732 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.250678    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.252422 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.252396    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.252827 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.252803    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.257616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.257600    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqzkr\" (UniqueName: \"kubernetes.io/projected/8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6-kube-api-access-jqzkr\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7\" (UID: \"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.350932 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.350839    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:01.485962 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:01.485935    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"]
Jul 06 20:35:01.488195 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:35:01.488164    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d1c9f29_a3b2_403a_a663_1d8e9ff1cfa6.slice/crio-ae23a159af45cabf5690b68a5b13f24d56860ff2e812a74efc20d199247ee8ee WatchSource:0}: Error finding container ae23a159af45cabf5690b68a5b13f24d56860ff2e812a74efc20d199247ee8ee: Status 404 returned error can't find the container with id ae23a159af45cabf5690b68a5b13f24d56860ff2e812a74efc20d199247ee8ee
Jul 06 20:35:02.022640 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:02.022598    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7" event={"ID":"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6","Type":"ContainerStarted","Data":"ae23a159af45cabf5690b68a5b13f24d56860ff2e812a74efc20d199247ee8ee"}
Jul 06 20:35:03.028154 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:03.028119    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7" event={"ID":"8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6","Type":"ContainerStarted","Data":"f6850fa5f9f8bd65413b9a3b1a7e6b1a5d2a678455eb64f237c1707676430539"}
Jul 06 20:35:03.047047 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:03.046986    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7" podStartSLOduration=0.908309891 podStartE2EDuration="2.046967546s" totalImagesPullingTime="1.138657655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:35:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:35:03.045617 +0000 UTC m=+704.329667540" watchObservedRunningTime="2026-07-06 20:35:03.046967546 +0000 UTC m=+704.331018150"
Jul 06 20:35:11.351186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:11.351094    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:11.351186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:11.351160    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:11.363702 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:11.363676    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:12.081299 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:12.081263    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7"
Jul 06 20:35:16.729268 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.729234    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"]
Jul 06 20:35:16.733942 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.733918    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.736781 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.736756    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Jul 06 20:35:16.743155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.743128    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"]
Jul 06 20:35:16.895994 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.895962    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.895994 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.896003    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.896267 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.896056    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgzck\" (UniqueName: \"kubernetes.io/projected/969dbebc-f597-4d81-8f58-35f240ad9f53-kube-api-access-fgzck\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.896267 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.896116    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/969dbebc-f597-4d81-8f58-35f240ad9f53-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.896267 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.896177    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.896267 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.896209    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997448    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fgzck\" (UniqueName: \"kubernetes.io/projected/969dbebc-f597-4d81-8f58-35f240ad9f53-kube-api-access-fgzck\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997505    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/969dbebc-f597-4d81-8f58-35f240ad9f53-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997552    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997575    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997648    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.997844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997684    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.998006 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.997911    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.998101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.998068    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.998101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.998082    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:16.999943 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:16.999911    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/969dbebc-f597-4d81-8f58-35f240ad9f53-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:17.000216 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:17.000198    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/969dbebc-f597-4d81-8f58-35f240ad9f53-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:17.006275 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:17.006251    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgzck\" (UniqueName: \"kubernetes.io/projected/969dbebc-f597-4d81-8f58-35f240ad9f53-kube-api-access-fgzck\") pod \"e2e-trlp-test-simulated-kserve-598466548d-h29gh\" (UID: \"969dbebc-f597-4d81-8f58-35f240ad9f53\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:17.045460 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:17.045419    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:17.177770 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:17.177742    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"]
Jul 06 20:35:17.180206 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:35:17.180182    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod969dbebc_f597_4d81_8f58_35f240ad9f53.slice/crio-0133210f7587a852b1d09e2f3f72afe5b4ee4c1b4e20be67c39d3f529503db53 WatchSource:0}: Error finding container 0133210f7587a852b1d09e2f3f72afe5b4ee4c1b4e20be67c39d3f529503db53: Status 404 returned error can't find the container with id 0133210f7587a852b1d09e2f3f72afe5b4ee4c1b4e20be67c39d3f529503db53
Jul 06 20:35:17.181443 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:17.181428    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:35:18.103464 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:18.103425    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh" event={"ID":"969dbebc-f597-4d81-8f58-35f240ad9f53","Type":"ContainerStarted","Data":"f5615ea625d4c45c6cd32f60f06d34b189a2ab8a7425be0260fb1c14b18a4b61"}
Jul 06 20:35:18.103852 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:18.103485    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh" event={"ID":"969dbebc-f597-4d81-8f58-35f240ad9f53","Type":"ContainerStarted","Data":"0133210f7587a852b1d09e2f3f72afe5b4ee4c1b4e20be67c39d3f529503db53"}
Jul 06 20:35:18.124054 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:18.123996    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh" podStartSLOduration=1.923387617 podStartE2EDuration="2.123980042s" totalImagesPullingTime="200.592425ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:35:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:35:18.123001889 +0000 UTC m=+719.407052434" watchObservedRunningTime="2026-07-06 20:35:18.123980042 +0000 UTC m=+719.408030583"
Jul 06 20:35:27.046082 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:27.046025    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:27.046848 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:27.046110    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:27.058538 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:27.058503    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:27.151984 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:27.151951    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-h29gh"
Jul 06 20:35:43.041944 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.041910    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"]
Jul 06 20:35:43.048330 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.048313    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.050948 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.050931    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 06 20:35:43.056192 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.056173    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"]
Jul 06 20:35:43.116609 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116570    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.116609 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116608    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67a6dc92-9b90-48cf-9458-b264c5426fab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.116818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116644    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.116818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116664    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.116818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116735    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnmng\" (UniqueName: \"kubernetes.io/projected/67a6dc92-9b90-48cf-9458-b264c5426fab-kube-api-access-fnmng\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.116818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.116769    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217650 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217621    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217657    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217683    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnmng\" (UniqueName: \"kubernetes.io/projected/67a6dc92-9b90-48cf-9458-b264c5426fab-kube-api-access-fnmng\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217708    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217786    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.217833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.217807    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67a6dc92-9b90-48cf-9458-b264c5426fab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.218155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.218061    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.218155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.218110    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.218268 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.218247    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.219911 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.219885    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67a6dc92-9b90-48cf-9458-b264c5426fab-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.220257 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.220240    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67a6dc92-9b90-48cf-9458-b264c5426fab-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.226308 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.226280    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnmng\" (UniqueName: \"kubernetes.io/projected/67a6dc92-9b90-48cf-9458-b264c5426fab-kube-api-access-fnmng\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-njk76\" (UID: \"67a6dc92-9b90-48cf-9458-b264c5426fab\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.359695 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.359621    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:43.488259 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:43.488231    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"]
Jul 06 20:35:43.490264 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:35:43.490232    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67a6dc92_9b90_48cf_9458_b264c5426fab.slice/crio-f84b175f06d91654e410461878da83217f8e222ea9b95492f535649fb76902ad WatchSource:0}: Error finding container f84b175f06d91654e410461878da83217f8e222ea9b95492f535649fb76902ad: Status 404 returned error can't find the container with id f84b175f06d91654e410461878da83217f8e222ea9b95492f535649fb76902ad
Jul 06 20:35:44.212654 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:44.212621    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76" event={"ID":"67a6dc92-9b90-48cf-9458-b264c5426fab","Type":"ContainerStarted","Data":"71fe96e75a16a76771fa36ea9a9f63b5103a862d7571a8591122914461193ac4"}
Jul 06 20:35:44.213040 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:44.212675    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76" event={"ID":"67a6dc92-9b90-48cf-9458-b264c5426fab","Type":"ContainerStarted","Data":"f84b175f06d91654e410461878da83217f8e222ea9b95492f535649fb76902ad"}
Jul 06 20:35:44.248618 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:44.248560    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76" podStartSLOduration=1.052248832 podStartE2EDuration="1.248542957s" totalImagesPullingTime="196.294125ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:35:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:35:44.246595788 +0000 UTC m=+745.530646340" watchObservedRunningTime="2026-07-06 20:35:44.248542957 +0000 UTC m=+745.532593508"
Jul 06 20:35:53.360085 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:53.360049    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:53.360587 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:53.360104    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:53.372872 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:53.372850    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:35:54.267799 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:35:54.267773    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-njk76"
Jul 06 20:38:19.161508 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:19.161438    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:38:19.161508 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:19.161493    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:38:44.761745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:44.761711    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:38:44.762245 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:44.761909    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-846c44d545-f5ztx" podUID="bbf66be7-d892-4698-a22c-da62968941c9" containerName="manager" containerID="cri-o://f9b3b192245b66b459ce557fe582e1a760168b70e916db60cdd07ce8663324ff" gracePeriod=30
Jul 06 20:38:44.995283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:44.995255    2554 generic.go:356] "Generic (PLEG): container finished" podID="bbf66be7-d892-4698-a22c-da62968941c9" containerID="f9b3b192245b66b459ce557fe582e1a760168b70e916db60cdd07ce8663324ff" exitCode=0
Jul 06 20:38:44.995407 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:44.995298    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-f5ztx" event={"ID":"bbf66be7-d892-4698-a22c-da62968941c9","Type":"ContainerDied","Data":"f9b3b192245b66b459ce557fe582e1a760168b70e916db60cdd07ce8663324ff"}
Jul 06 20:38:45.011888 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.011832    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:38:45.041334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.041308    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2gtr\" (UniqueName: \"kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr\") pod \"bbf66be7-d892-4698-a22c-da62968941c9\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") "
Jul 06 20:38:45.041491 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.041399    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert\") pod \"bbf66be7-d892-4698-a22c-da62968941c9\" (UID: \"bbf66be7-d892-4698-a22c-da62968941c9\") "
Jul 06 20:38:45.043737 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.043707    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr" (OuterVolumeSpecName: "kube-api-access-q2gtr") pod "bbf66be7-d892-4698-a22c-da62968941c9" (UID: "bbf66be7-d892-4698-a22c-da62968941c9"). InnerVolumeSpecName "kube-api-access-q2gtr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:38:45.044032 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.044008    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "bbf66be7-d892-4698-a22c-da62968941c9" (UID: "bbf66be7-d892-4698-a22c-da62968941c9"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:38:45.142553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.142504    2554 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bbf66be7-d892-4698-a22c-da62968941c9-webhook-cert\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:38:45.142553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:45.142556    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2gtr\" (UniqueName: \"kubernetes.io/projected/bbf66be7-d892-4698-a22c-da62968941c9-kube-api-access-q2gtr\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:38:46.000730 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.000684    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-f5ztx" event={"ID":"bbf66be7-d892-4698-a22c-da62968941c9","Type":"ContainerDied","Data":"5ccd8e24fc4c884356bde18f4c0ee44009e4fea50581572db28de8368bb603d0"}
Jul 06 20:38:46.000730 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.000733    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-f5ztx"
Jul 06 20:38:46.001186 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.000817    2554 scope.go:117] "RemoveContainer" containerID="f9b3b192245b66b459ce557fe582e1a760168b70e916db60cdd07ce8663324ff"
Jul 06 20:38:46.032392 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.032359    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:38:46.035664 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.035637    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-846c44d545-f5ztx"]
Jul 06 20:38:46.850692 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.850662    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-846c44d545-sfr4d"]
Jul 06 20:38:46.850999 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.850988    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf66be7-d892-4698-a22c-da62968941c9" containerName="manager"
Jul 06 20:38:46.851047 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.851002    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf66be7-d892-4698-a22c-da62968941c9" containerName="manager"
Jul 06 20:38:46.851085 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.851063    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf66be7-d892-4698-a22c-da62968941c9" containerName="manager"
Jul 06 20:38:46.855187 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.855162    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:46.858568 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.858520    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 06 20:38:46.859603 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.859587    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2qjmv\""
Jul 06 20:38:46.859699 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.859588    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 06 20:38:46.865251 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.865225    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-846c44d545-sfr4d"]
Jul 06 20:38:46.956164 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.956123    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lst6f\" (UniqueName: \"kubernetes.io/projected/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-kube-api-access-lst6f\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:46.956336 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:46.956221    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-webhook-cert\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.057329 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.057302    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lst6f\" (UniqueName: \"kubernetes.io/projected/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-kube-api-access-lst6f\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.057737 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.057359    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-webhook-cert\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.060032 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.060007    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-webhook-cert\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.066381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.066354    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lst6f\" (UniqueName: \"kubernetes.io/projected/c1e638d5-bce8-43e7-b1fe-1879e4f02fe4-kube-api-access-lst6f\") pod \"maas-controller-846c44d545-sfr4d\" (UID: \"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4\") " pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.166955 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.166868    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:47.283913 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.283879    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf66be7-d892-4698-a22c-da62968941c9" path="/var/lib/kubelet/pods/bbf66be7-d892-4698-a22c-da62968941c9/volumes"
Jul 06 20:38:47.303987 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:47.303967    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-846c44d545-sfr4d"]
Jul 06 20:38:47.306139 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:38:47.306111    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1e638d5_bce8_43e7_b1fe_1879e4f02fe4.slice/crio-ddc636bc23e4920bec3b5b9475fd9f8bf5925d6715ff4e7c455375bc50605479 WatchSource:0}: Error finding container ddc636bc23e4920bec3b5b9475fd9f8bf5925d6715ff4e7c455375bc50605479: Status 404 returned error can't find the container with id ddc636bc23e4920bec3b5b9475fd9f8bf5925d6715ff4e7c455375bc50605479
Jul 06 20:38:48.012874 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:48.012833    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-sfr4d" event={"ID":"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4","Type":"ContainerStarted","Data":"769cee8d08e24978aa558c35647e6eb42774f45b22874938d100e26618abb0d1"}
Jul 06 20:38:48.013058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:48.012891    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-846c44d545-sfr4d" event={"ID":"c1e638d5-bce8-43e7-b1fe-1879e4f02fe4","Type":"ContainerStarted","Data":"ddc636bc23e4920bec3b5b9475fd9f8bf5925d6715ff4e7c455375bc50605479"}
Jul 06 20:38:48.013058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:48.012981    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:38:48.032736 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:48.032686    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-846c44d545-sfr4d" podStartSLOduration=1.638167156 podStartE2EDuration="2.03266986s" totalImagesPullingTime="394.502704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:38:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:38:48.032322484 +0000 UTC m=+929.316373025" watchObservedRunningTime="2026-07-06 20:38:48.03266986 +0000 UTC m=+929.316720400"
Jul 06 20:38:59.022786 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:38:59.022751    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-846c44d545-sfr4d"
Jul 06 20:43:19.187754 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:43:19.187726    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:43:19.188337 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:43:19.188036    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:44:01.551270 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:01.551236    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:44:01.551904 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:01.551504    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" podUID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" containerName="manager" containerID="cri-o://8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661" gracePeriod=10
Jul 06 20:44:04.000128 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.000103    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:44:04.111285 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.111197    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume\") pod \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") "
Jul 06 20:44:04.111285 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.111246    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mxpk\" (UniqueName: \"kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk\") pod \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\" (UID: \"c2ad77db-b1cb-4289-9e7a-1f5092d536dd\") "
Jul 06 20:44:04.111591 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.111561    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c2ad77db-b1cb-4289-9e7a-1f5092d536dd" (UID: "c2ad77db-b1cb-4289-9e7a-1f5092d536dd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 20:44:04.113247 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.113219    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk" (OuterVolumeSpecName: "kube-api-access-5mxpk") pod "c2ad77db-b1cb-4289-9e7a-1f5092d536dd" (UID: "c2ad77db-b1cb-4289-9e7a-1f5092d536dd"). InnerVolumeSpecName "kube-api-access-5mxpk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:44:04.211954 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.211903    2554 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-extensions-socket-volume\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:44:04.211954 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.211947    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mxpk\" (UniqueName: \"kubernetes.io/projected/c2ad77db-b1cb-4289-9e7a-1f5092d536dd-kube-api-access-5mxpk\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:44:04.412445 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.412342    2554 generic.go:356] "Generic (PLEG): container finished" podID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" containerID="8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661" exitCode=0
Jul 06 20:44:04.412445 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.412425    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"
Jul 06 20:44:04.412445 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.412431    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" event={"ID":"c2ad77db-b1cb-4289-9e7a-1f5092d536dd","Type":"ContainerDied","Data":"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"}
Jul 06 20:44:04.412762 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.412488    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv" event={"ID":"c2ad77db-b1cb-4289-9e7a-1f5092d536dd","Type":"ContainerDied","Data":"bf06ff6a666e682add06bf9736ef6f74098d29780ee6a1a8840295eb47ec6834"}
Jul 06 20:44:04.412762 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.412511    2554 scope.go:117] "RemoveContainer" containerID="8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"
Jul 06 20:44:04.423019 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.423003    2554 scope.go:117] "RemoveContainer" containerID="8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"
Jul 06 20:44:04.423277 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:44:04.423260    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661\": container with ID starting with 8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661 not found: ID does not exist" containerID="8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"
Jul 06 20:44:04.423323 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.423287    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661"} err="failed to get container status \"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661\": rpc error: code = NotFound desc = could not find container \"8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661\": container with ID starting with 8e0dffaeee992afeec4c22f9af5c87f214e0c495130e20233c185d1c63ee0661 not found: ID does not exist"
Jul 06 20:44:04.443009 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.442973    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:44:04.457800 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:04.457768    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-cpbxv"]
Jul 06 20:44:05.281204 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:44:05.281168    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" path="/var/lib/kubelet/pods/c2ad77db-b1cb-4289-9e7a-1f5092d536dd/volumes"
Jul 06 20:45:00.163649 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.163614    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-zlc7l"]
Jul 06 20:45:00.164230 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.163934    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" containerName="manager"
Jul 06 20:45:00.164230 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.163944    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" containerName="manager"
Jul 06 20:45:00.164230 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.164009    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ad77db-b1cb-4289-9e7a-1f5092d536dd" containerName="manager"
Jul 06 20:45:00.167096 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.167078    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:00.182697 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.182669    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-zlc7l"]
Jul 06 20:45:00.286370 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.286342    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2vfx\" (UniqueName: \"kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx\") pod \"maas-api-key-cleanup-29722845-zlc7l\" (UID: \"5812717c-71a0-41a7-8d35-a0fa2b712aaa\") " pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:00.387768 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.387725    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x2vfx\" (UniqueName: \"kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx\") pod \"maas-api-key-cleanup-29722845-zlc7l\" (UID: \"5812717c-71a0-41a7-8d35-a0fa2b712aaa\") " pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:00.397280 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.397256    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2vfx\" (UniqueName: \"kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx\") pod \"maas-api-key-cleanup-29722845-zlc7l\" (UID: \"5812717c-71a0-41a7-8d35-a0fa2b712aaa\") " pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:00.477347 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.477243    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:00.610940 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.610914    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722845-zlc7l"]
Jul 06 20:45:00.612975 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:45:00.612948    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5812717c_71a0_41a7_8d35_a0fa2b712aaa.slice/crio-f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec WatchSource:0}: Error finding container f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec: Status 404 returned error can't find the container with id f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec
Jul 06 20:45:00.614709 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.614691    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:45:00.659689 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:00.659652    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l" event={"ID":"5812717c-71a0-41a7-8d35-a0fa2b712aaa","Type":"ContainerStarted","Data":"f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec"}
Jul 06 20:45:02.874859 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:02.874828    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"]
Jul 06 20:45:02.879220 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:02.879201    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:02.882006 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:02.881984    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6qt66\""
Jul 06 20:45:02.897248 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:02.897221    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"]
Jul 06 20:45:03.014141 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.014096    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2efbde4a-6867-4c5e-be6a-782ad29a5a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.014328 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.014153    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcxcq\" (UniqueName: \"kubernetes.io/projected/2efbde4a-6867-4c5e-be6a-782ad29a5a61-kube-api-access-fcxcq\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.115290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.115248    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2efbde4a-6867-4c5e-be6a-782ad29a5a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.115290 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.115296    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcxcq\" (UniqueName: \"kubernetes.io/projected/2efbde4a-6867-4c5e-be6a-782ad29a5a61-kube-api-access-fcxcq\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.115686 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.115668    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2efbde4a-6867-4c5e-be6a-782ad29a5a61-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.129507 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.129484    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcxcq\" (UniqueName: \"kubernetes.io/projected/2efbde4a-6867-4c5e-be6a-782ad29a5a61-kube-api-access-fcxcq\") pod \"kuadrant-operator-controller-manager-86878c769f-gxw2w\" (UID: \"2efbde4a-6867-4c5e-be6a-782ad29a5a61\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.189272 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.189218    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.332213 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.332189    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"]
Jul 06 20:45:03.333778 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:45:03.333753    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2efbde4a_6867_4c5e_be6a_782ad29a5a61.slice/crio-628201693cbde9709c1da2fcd0bcb3bd74649f7bee0256f4dc3ef37dfb18bd2f WatchSource:0}: Error finding container 628201693cbde9709c1da2fcd0bcb3bd74649f7bee0256f4dc3ef37dfb18bd2f: Status 404 returned error can't find the container with id 628201693cbde9709c1da2fcd0bcb3bd74649f7bee0256f4dc3ef37dfb18bd2f
Jul 06 20:45:03.677140 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.677100    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w" event={"ID":"2efbde4a-6867-4c5e-be6a-782ad29a5a61","Type":"ContainerStarted","Data":"0a915dc600f60beebdc893f16ddee8cfb96101dfe1f26b5220c68d80ef13e782"}
Jul 06 20:45:03.677343 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.677169    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w" event={"ID":"2efbde4a-6867-4c5e-be6a-782ad29a5a61","Type":"ContainerStarted","Data":"628201693cbde9709c1da2fcd0bcb3bd74649f7bee0256f4dc3ef37dfb18bd2f"}
Jul 06 20:45:03.677343 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.677215    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:45:03.678719 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.678697    2554 generic.go:356] "Generic (PLEG): container finished" podID="5812717c-71a0-41a7-8d35-a0fa2b712aaa" containerID="9122ec9e4fa4b95c427569da89aa5ea6da20cdb6ebd6597b00edff85e163dfc4" exitCode=0
Jul 06 20:45:03.678834 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.678748    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l" event={"ID":"5812717c-71a0-41a7-8d35-a0fa2b712aaa","Type":"ContainerDied","Data":"9122ec9e4fa4b95c427569da89aa5ea6da20cdb6ebd6597b00edff85e163dfc4"}
Jul 06 20:45:03.696782 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:03.696735    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w" podStartSLOduration=1.6967183380000002 podStartE2EDuration="1.696718338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:45:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:45:03.695579073 +0000 UTC m=+1304.979629615" watchObservedRunningTime="2026-07-06 20:45:03.696718338 +0000 UTC m=+1304.980768878"
Jul 06 20:45:04.812283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:04.812261    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:04.931208 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:04.931171    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2vfx\" (UniqueName: \"kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx\") pod \"5812717c-71a0-41a7-8d35-a0fa2b712aaa\" (UID: \"5812717c-71a0-41a7-8d35-a0fa2b712aaa\") "
Jul 06 20:45:04.933255 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:04.933214    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx" (OuterVolumeSpecName: "kube-api-access-x2vfx") pod "5812717c-71a0-41a7-8d35-a0fa2b712aaa" (UID: "5812717c-71a0-41a7-8d35-a0fa2b712aaa"). InnerVolumeSpecName "kube-api-access-x2vfx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:45:05.032393 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:05.032350    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2vfx\" (UniqueName: \"kubernetes.io/projected/5812717c-71a0-41a7-8d35-a0fa2b712aaa-kube-api-access-x2vfx\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:45:05.689119 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:05.689040    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l"
Jul 06 20:45:05.689119 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:05.689074    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722845-zlc7l" event={"ID":"5812717c-71a0-41a7-8d35-a0fa2b712aaa","Type":"ContainerDied","Data":"f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec"}
Jul 06 20:45:05.689119 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:05.689118    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f729f1ce82762fbf83fc0395b9d16a4291a6bd0d4fe630e847997ba56cf342ec"
Jul 06 20:45:14.685314 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:45:14.685278    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-gxw2w"
Jul 06 20:48:19.212838 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:48:19.212811    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:48:19.215217 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:48:19.215195    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:51:23.117030 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.116998    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:23.117552 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.117499    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5812717c-71a0-41a7-8d35-a0fa2b712aaa" containerName="cleanup"
Jul 06 20:51:23.117552 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.117518    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5812717c-71a0-41a7-8d35-a0fa2b712aaa" containerName="cleanup"
Jul 06 20:51:23.117683 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.117619    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5812717c-71a0-41a7-8d35-a0fa2b712aaa" containerName="cleanup"
Jul 06 20:51:23.120514 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.120493    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.123612 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.123596    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b01900e5-serving-cert\""
Jul 06 20:51:23.137289 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.137267    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:23.225855 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.225819    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.226012 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.225898    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vv8hp\" (UniqueName: \"kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.226012 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.225926    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.327009 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.326978    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.327196 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.327037    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vv8hp\" (UniqueName: \"kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.327196 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.327061    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.327312 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:23.327295    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b01900e5-serving-cert: secret "maas-api-e2e-ait-b01900e5-serving-cert" not found
Jul 06 20:51:23.327373 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:23.327353    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls podName:60d92311-cf3d-41bd-bb29-c656c16cf57a nodeName:}" failed. No retries permitted until 2026-07-06 20:51:23.827337822 +0000 UTC m=+1685.111388345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls") pod "maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" (UID: "60d92311-cf3d-41bd-bb29-c656c16cf57a") : secret "maas-api-e2e-ait-b01900e5-serving-cert" not found
Jul 06 20:51:23.327832 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.327807    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.336963 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.336938    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vv8hp\" (UniqueName: \"kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.831577 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.831542    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:23.834127 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:23.834100    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") pod \"maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") " pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:24.031616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:24.031569    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:24.173936 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:24.173904    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:24.178539 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:51:24.178504    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60d92311_cf3d_41bd_bb29_c656c16cf57a.slice/crio-b821da1409f0c4473e976e60f6f491f6a293f683f7b049435401c9e8d3276855 WatchSource:0}: Error finding container b821da1409f0c4473e976e60f6f491f6a293f683f7b049435401c9e8d3276855: Status 404 returned error can't find the container with id b821da1409f0c4473e976e60f6f491f6a293f683f7b049435401c9e8d3276855
Jul 06 20:51:24.179942 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:24.179916    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:51:24.387836 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:24.387742    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" event={"ID":"60d92311-cf3d-41bd-bb29-c656c16cf57a","Type":"ContainerStarted","Data":"b821da1409f0c4473e976e60f6f491f6a293f683f7b049435401c9e8d3276855"}
Jul 06 20:51:25.393899 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:25.393854    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" event={"ID":"60d92311-cf3d-41bd-bb29-c656c16cf57a","Type":"ContainerStarted","Data":"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"}
Jul 06 20:51:25.394344 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:25.393936    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:25.414703 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:25.414651    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" podStartSLOduration=1.939213231 podStartE2EDuration="2.41463369s" totalImagesPullingTime="475.420459ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:51:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:51:25.413594212 +0000 UTC m=+1686.697644755" watchObservedRunningTime="2026-07-06 20:51:25.41463369 +0000 UTC m=+1686.698684231"
Jul 06 20:51:28.661750 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.661719    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:28.662205 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.661911    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" podUID="60d92311-cf3d-41bd-bb29-c656c16cf57a" containerName="maas-api" containerID="cri-o://35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b" gracePeriod=30
Jul 06 20:51:28.898563 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.898520    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:28.969514 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.969437    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") pod \"60d92311-cf3d-41bd-bb29-c656c16cf57a\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") "
Jul 06 20:51:28.969514 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.969476    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca\") pod \"60d92311-cf3d-41bd-bb29-c656c16cf57a\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") "
Jul 06 20:51:28.969514 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.969493    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vv8hp\" (UniqueName: \"kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp\") pod \"60d92311-cf3d-41bd-bb29-c656c16cf57a\" (UID: \"60d92311-cf3d-41bd-bb29-c656c16cf57a\") "
Jul 06 20:51:28.969853 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.969832    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60d92311-cf3d-41bd-bb29-c656c16cf57a" (UID: "60d92311-cf3d-41bd-bb29-c656c16cf57a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:28.971638 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.971609    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60d92311-cf3d-41bd-bb29-c656c16cf57a" (UID: "60d92311-cf3d-41bd-bb29-c656c16cf57a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:28.971755 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:28.971634    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp" (OuterVolumeSpecName: "kube-api-access-vv8hp") pod "60d92311-cf3d-41bd-bb29-c656c16cf57a" (UID: "60d92311-cf3d-41bd-bb29-c656c16cf57a"). InnerVolumeSpecName "kube-api-access-vv8hp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:29.070307 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.070264    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60d92311-cf3d-41bd-bb29-c656c16cf57a-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:29.070307 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.070299    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vv8hp\" (UniqueName: \"kubernetes.io/projected/60d92311-cf3d-41bd-bb29-c656c16cf57a-kube-api-access-vv8hp\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:29.070307 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.070310    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60d92311-cf3d-41bd-bb29-c656c16cf57a-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:29.413873 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.413833    2554 generic.go:356] "Generic (PLEG): container finished" podID="60d92311-cf3d-41bd-bb29-c656c16cf57a" containerID="35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b" exitCode=0
Jul 06 20:51:29.414012 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.413883    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"
Jul 06 20:51:29.414012 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.413884    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" event={"ID":"60d92311-cf3d-41bd-bb29-c656c16cf57a","Type":"ContainerDied","Data":"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"}
Jul 06 20:51:29.414108 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.414014    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n" event={"ID":"60d92311-cf3d-41bd-bb29-c656c16cf57a","Type":"ContainerDied","Data":"b821da1409f0c4473e976e60f6f491f6a293f683f7b049435401c9e8d3276855"}
Jul 06 20:51:29.414108 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.414048    2554 scope.go:117] "RemoveContainer" containerID="35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"
Jul 06 20:51:29.422439 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.422356    2554 scope.go:117] "RemoveContainer" containerID="35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"
Jul 06 20:51:29.422847 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:29.422819    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b\": container with ID starting with 35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b not found: ID does not exist" containerID="35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"
Jul 06 20:51:29.422960 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.422857    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b"} err="failed to get container status \"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b\": rpc error: code = NotFound desc = could not find container \"35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b\": container with ID starting with 35006a3c31ffc8e97f921ec49b55dfa99c96ed0a18b2e8a9854118f04086a44b not found: ID does not exist"
Jul 06 20:51:29.435699 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.435677    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:29.440615 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:29.440596    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b01900e5-69d877f88c-q7l7n"]
Jul 06 20:51:31.286764 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:31.286734    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d92311-cf3d-41bd-bb29-c656c16cf57a" path="/var/lib/kubelet/pods/60d92311-cf3d-41bd-bb29-c656c16cf57a/volumes"
Jul 06 20:51:35.821335 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.821302    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:35.821729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.821633    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60d92311-cf3d-41bd-bb29-c656c16cf57a" containerName="maas-api"
Jul 06 20:51:35.821729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.821644    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="60d92311-cf3d-41bd-bb29-c656c16cf57a" containerName="maas-api"
Jul 06 20:51:35.821729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.821705    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="60d92311-cf3d-41bd-bb29-c656c16cf57a" containerName="maas-api"
Jul 06 20:51:35.826141 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.826113    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:35.828630 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.828607    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-b097e72f-serving-cert\""
Jul 06 20:51:35.839838 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.839811    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:35.920003 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.919959    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:35.920003 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.920011    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xpl9\" (UniqueName: \"kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:35.920218 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:35.920039    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.021163 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.021126    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xpl9\" (UniqueName: \"kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.021334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.021174    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.021334 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.021260    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.021412 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:36.021356    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b097e72f-serving-cert: secret "maas-api-e2e-ait-b097e72f-serving-cert" not found
Jul 06 20:51:36.021448 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:36.021419    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls podName:bdab3980-21fb-4fe5-ad10-69dbd49afb4b nodeName:}" failed. No retries permitted until 2026-07-06 20:51:36.521399362 +0000 UTC m=+1697.805449881 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls") pod "maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" (UID: "bdab3980-21fb-4fe5-ad10-69dbd49afb4b") : secret "maas-api-e2e-ait-b097e72f-serving-cert" not found
Jul 06 20:51:36.021820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.021801    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.031693 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.031671    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xpl9\" (UniqueName: \"kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.525940 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:36.525896    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:36.526135 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:36.526024    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-ait-b097e72f-serving-cert: secret "maas-api-e2e-ait-b097e72f-serving-cert" not found
Jul 06 20:51:36.526135 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:36.526083    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls podName:bdab3980-21fb-4fe5-ad10-69dbd49afb4b nodeName:}" failed. No retries permitted until 2026-07-06 20:51:37.526064901 +0000 UTC m=+1698.810115427 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls") pod "maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" (UID: "bdab3980-21fb-4fe5-ad10-69dbd49afb4b") : secret "maas-api-e2e-ait-b097e72f-serving-cert" not found
Jul 06 20:51:37.536466 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:37.536425    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:37.539112 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:37.539089    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") " pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:37.638668 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:37.638628    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:37.775005 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:37.774978    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:37.777712 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:51:37.777677    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdab3980_21fb_4fe5_ad10_69dbd49afb4b.slice/crio-a848c47fd0c9918582141b87f22da7990f29c1307e605a83ef0404c1063f091c WatchSource:0}: Error finding container a848c47fd0c9918582141b87f22da7990f29c1307e605a83ef0404c1063f091c: Status 404 returned error can't find the container with id a848c47fd0c9918582141b87f22da7990f29c1307e605a83ef0404c1063f091c
Jul 06 20:51:38.458125 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:38.458030    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" event={"ID":"bdab3980-21fb-4fe5-ad10-69dbd49afb4b","Type":"ContainerStarted","Data":"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"}
Jul 06 20:51:38.458125 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:38.458123    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:38.458317 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:38.458144    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" event={"ID":"bdab3980-21fb-4fe5-ad10-69dbd49afb4b","Type":"ContainerStarted","Data":"a848c47fd0c9918582141b87f22da7990f29c1307e605a83ef0404c1063f091c"}
Jul 06 20:51:38.476468 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:38.476424    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" podStartSLOduration=3.1455055 podStartE2EDuration="3.476410198s" totalImagesPullingTime="330.904698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:51:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:51:38.474640011 +0000 UTC m=+1699.758690552" watchObservedRunningTime="2026-07-06 20:51:38.476410198 +0000 UTC m=+1699.760460738"
Jul 06 20:51:41.181391 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.181360    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:41.181859 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.181629    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" podUID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" containerName="maas-api" containerID="cri-o://95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057" gracePeriod=30
Jul 06 20:51:41.418098 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.418076    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:41.468958 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.468877    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca\") pod \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") "
Jul 06 20:51:41.469120 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.468966    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xpl9\" (UniqueName: \"kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9\") pod \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") "
Jul 06 20:51:41.469120 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.469008    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") pod \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\" (UID: \"bdab3980-21fb-4fe5-ad10-69dbd49afb4b\") "
Jul 06 20:51:41.469296 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.469271    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bdab3980-21fb-4fe5-ad10-69dbd49afb4b" (UID: "bdab3980-21fb-4fe5-ad10-69dbd49afb4b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:41.471202 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.471171    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9" (OuterVolumeSpecName: "kube-api-access-5xpl9") pod "bdab3980-21fb-4fe5-ad10-69dbd49afb4b" (UID: "bdab3980-21fb-4fe5-ad10-69dbd49afb4b"). InnerVolumeSpecName "kube-api-access-5xpl9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:41.471523 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.471499    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bdab3980-21fb-4fe5-ad10-69dbd49afb4b" (UID: "bdab3980-21fb-4fe5-ad10-69dbd49afb4b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:41.474357 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.474330    2554 generic.go:356] "Generic (PLEG): container finished" podID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" containerID="95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057" exitCode=0
Jul 06 20:51:41.474492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.474379    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"
Jul 06 20:51:41.474492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.474410    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" event={"ID":"bdab3980-21fb-4fe5-ad10-69dbd49afb4b","Type":"ContainerDied","Data":"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"}
Jul 06 20:51:41.474492 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.474469    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45" event={"ID":"bdab3980-21fb-4fe5-ad10-69dbd49afb4b","Type":"ContainerDied","Data":"a848c47fd0c9918582141b87f22da7990f29c1307e605a83ef0404c1063f091c"}
Jul 06 20:51:41.474687 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.474497    2554 scope.go:117] "RemoveContainer" containerID="95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"
Jul 06 20:51:41.488969 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.488950    2554 scope.go:117] "RemoveContainer" containerID="95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"
Jul 06 20:51:41.489285 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:41.489268    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057\": container with ID starting with 95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057 not found: ID does not exist" containerID="95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"
Jul 06 20:51:41.489349 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.489299    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057"} err="failed to get container status \"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057\": rpc error: code = NotFound desc = could not find container \"95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057\": container with ID starting with 95ed2eddce22417907b0ce09c88017e52f84a77dca5fde87ebe1ab5c78d64057 not found: ID does not exist"
Jul 06 20:51:41.499721 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.499693    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:41.503565 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.503542    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-b097e72f-7db946b8d7-xgf45"]
Jul 06 20:51:41.570433 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.570394    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:41.570433 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.570424    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xpl9\" (UniqueName: \"kubernetes.io/projected/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-kube-api-access-5xpl9\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:41.570433 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:41.570436    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bdab3980-21fb-4fe5-ad10-69dbd49afb4b-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:43.280900 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:43.280867    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" path="/var/lib/kubelet/pods/bdab3980-21fb-4fe5-ad10-69dbd49afb4b/volumes"
Jul 06 20:51:54.565999 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.565967    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:51:54.566560 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.566291    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" containerName="maas-api"
Jul 06 20:51:54.566560 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.566304    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" containerName="maas-api"
Jul 06 20:51:54.566560 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.566369    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="bdab3980-21fb-4fe5-ad10-69dbd49afb4b" containerName="maas-api"
Jul 06 20:51:54.569553 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.569519    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.572224 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.572198    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-0f2dbdef-serving-cert\""
Jul 06 20:51:54.581035 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.581010    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:51:54.685801 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.685763    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2lfq\" (UniqueName: \"kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.685801 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.685801    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.686054 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.685836    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.786700 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.786665    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2lfq\" (UniqueName: \"kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.786870 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.786708    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.786870 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.786760    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.786990 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:54.786919    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-0f2dbdef-serving-cert: secret "maas-api-e2e-derive-0f2dbdef-serving-cert" not found
Jul 06 20:51:54.787044 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:54.787011    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls podName:d395a3d0-090d-4e5f-8dad-6bb427a54363 nodeName:}" failed. No retries permitted until 2026-07-06 20:51:55.286975765 +0000 UTC m=+1716.571026297 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls") pod "maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" (UID: "d395a3d0-090d-4e5f-8dad-6bb427a54363") : secret "maas-api-e2e-derive-0f2dbdef-serving-cert" not found
Jul 06 20:51:54.787388 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.787366    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:54.799364 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:54.799336    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2lfq\" (UniqueName: \"kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:55.291943 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:55.291911    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:55.294554 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:55.294510    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") pod \"maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") " pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:55.481397 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:55.481352    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:55.632379 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:55.632348    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:51:55.634066 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:51:55.634041    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd395a3d0_090d_4e5f_8dad_6bb427a54363.slice/crio-4b825895cad822d30d1e7f34a761fc504f121c93100f4b50cd93fb4797074fa5 WatchSource:0}: Error finding container 4b825895cad822d30d1e7f34a761fc504f121c93100f4b50cd93fb4797074fa5: Status 404 returned error can't find the container with id 4b825895cad822d30d1e7f34a761fc504f121c93100f4b50cd93fb4797074fa5
Jul 06 20:51:56.547258 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:56.547216    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" event={"ID":"d395a3d0-090d-4e5f-8dad-6bb427a54363","Type":"ContainerStarted","Data":"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"}
Jul 06 20:51:56.547438 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:56.547295    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" event={"ID":"d395a3d0-090d-4e5f-8dad-6bb427a54363","Type":"ContainerStarted","Data":"4b825895cad822d30d1e7f34a761fc504f121c93100f4b50cd93fb4797074fa5"}
Jul 06 20:51:56.547438 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:56.547362    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:56.566788 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:56.566726    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" podStartSLOduration=2.191572366 podStartE2EDuration="2.566710154s" totalImagesPullingTime="375.137788ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:51:56.566124301 +0000 UTC m=+1717.850174845" watchObservedRunningTime="2026-07-06 20:51:56.566710154 +0000 UTC m=+1717.850760695"
Jul 06 20:51:59.232420 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.232380    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:51:59.232884 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.232611    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" podUID="d395a3d0-090d-4e5f-8dad-6bb427a54363" containerName="maas-api" containerID="cri-o://6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d" gracePeriod=30
Jul 06 20:51:59.494228 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.494171    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:59.561140 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.561104    2554 generic.go:356] "Generic (PLEG): container finished" podID="d395a3d0-090d-4e5f-8dad-6bb427a54363" containerID="6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d" exitCode=0
Jul 06 20:51:59.561140 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.561144    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"
Jul 06 20:51:59.561384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.561189    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" event={"ID":"d395a3d0-090d-4e5f-8dad-6bb427a54363","Type":"ContainerDied","Data":"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"}
Jul 06 20:51:59.561384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.561251    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8" event={"ID":"d395a3d0-090d-4e5f-8dad-6bb427a54363","Type":"ContainerDied","Data":"4b825895cad822d30d1e7f34a761fc504f121c93100f4b50cd93fb4797074fa5"}
Jul 06 20:51:59.561384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.561281    2554 scope.go:117] "RemoveContainer" containerID="6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"
Jul 06 20:51:59.569570 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.569555    2554 scope.go:117] "RemoveContainer" containerID="6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"
Jul 06 20:51:59.569839 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:51:59.569821    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d\": container with ID starting with 6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d not found: ID does not exist" containerID="6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"
Jul 06 20:51:59.569887 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.569853    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d"} err="failed to get container status \"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d\": rpc error: code = NotFound desc = could not find container \"6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d\": container with ID starting with 6e7ad590d063dfb2f7df94af0ff3d043d079a8f1be35fc3d2f29e0f767eca56d not found: ID does not exist"
Jul 06 20:51:59.633399 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.633365    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2lfq\" (UniqueName: \"kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq\") pod \"d395a3d0-090d-4e5f-8dad-6bb427a54363\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") "
Jul 06 20:51:59.633597 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.633405    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") pod \"d395a3d0-090d-4e5f-8dad-6bb427a54363\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") "
Jul 06 20:51:59.633597 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.633570    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca\") pod \"d395a3d0-090d-4e5f-8dad-6bb427a54363\" (UID: \"d395a3d0-090d-4e5f-8dad-6bb427a54363\") "
Jul 06 20:51:59.633926 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.633902    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d395a3d0-090d-4e5f-8dad-6bb427a54363" (UID: "d395a3d0-090d-4e5f-8dad-6bb427a54363"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:51:59.635525 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.635503    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq" (OuterVolumeSpecName: "kube-api-access-v2lfq") pod "d395a3d0-090d-4e5f-8dad-6bb427a54363" (UID: "d395a3d0-090d-4e5f-8dad-6bb427a54363"). InnerVolumeSpecName "kube-api-access-v2lfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:51:59.635525 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.635512    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d395a3d0-090d-4e5f-8dad-6bb427a54363" (UID: "d395a3d0-090d-4e5f-8dad-6bb427a54363"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:51:59.734693 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.734657    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d395a3d0-090d-4e5f-8dad-6bb427a54363-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:59.734693 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.734688    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2lfq\" (UniqueName: \"kubernetes.io/projected/d395a3d0-090d-4e5f-8dad-6bb427a54363-kube-api-access-v2lfq\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:59.734885 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.734702    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d395a3d0-090d-4e5f-8dad-6bb427a54363-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:51:59.882518 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.882489    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:51:59.886141 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:51:59.886115    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-0f2dbdef-654db4bf87-9vvq8"]
Jul 06 20:52:01.281925 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:52:01.281888    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d395a3d0-090d-4e5f-8dad-6bb427a54363" path="/var/lib/kubelet/pods/d395a3d0-090d-4e5f-8dad-6bb427a54363/volumes"
Jul 06 20:53:19.241701 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:19.241672    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:53:19.243697 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:19.243676    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:53:46.109813 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.109776    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:46.110315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.110249    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d395a3d0-090d-4e5f-8dad-6bb427a54363" containerName="maas-api"
Jul 06 20:53:46.110315 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.110267    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d395a3d0-090d-4e5f-8dad-6bb427a54363" containerName="maas-api"
Jul 06 20:53:46.110431 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.110377    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d395a3d0-090d-4e5f-8dad-6bb427a54363" containerName="maas-api"
Jul 06 20:53:46.113262 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.113243    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.116270 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.116247    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-58c7cdd8-serving-cert\""
Jul 06 20:53:46.135918 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.135883    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:46.252498 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.252465    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.252696 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.252542    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.252696 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.252582    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7w7n\" (UniqueName: \"kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.353105 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.353068    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.353283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.353122    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.353283 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.353162    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7w7n\" (UniqueName: \"kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.353367 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:53:46.353281    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-58c7cdd8-serving-cert: secret "maas-api-e2e-mt-58c7cdd8-serving-cert" not found
Jul 06 20:53:46.353367 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:53:46.353349    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls podName:75b46a51-454e-4f33-8229-75cc86f82b75 nodeName:}" failed. No retries permitted until 2026-07-06 20:53:46.853330524 +0000 UTC m=+1828.137381043 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls") pod "maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" (UID: "75b46a51-454e-4f33-8229-75cc86f82b75") : secret "maas-api-e2e-mt-58c7cdd8-serving-cert" not found
Jul 06 20:53:46.353826 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.353807    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.372007 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.371939    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7w7n\" (UniqueName: \"kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.858654 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.858606    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:46.861164 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:46.861133    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") pod \"maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") " pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:47.023752 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:47.023716    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:47.200160 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:47.200084    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:47.203267 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:53:47.203240    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75b46a51_454e_4f33_8229_75cc86f82b75.slice/crio-21af1c36f789d7db4b9365e469325936a2e99d791a5cc32f8a554949d6fd19a4 WatchSource:0}: Error finding container 21af1c36f789d7db4b9365e469325936a2e99d791a5cc32f8a554949d6fd19a4: Status 404 returned error can't find the container with id 21af1c36f789d7db4b9365e469325936a2e99d791a5cc32f8a554949d6fd19a4
Jul 06 20:53:48.056927 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:48.056891    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" event={"ID":"75b46a51-454e-4f33-8229-75cc86f82b75","Type":"ContainerStarted","Data":"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"}
Jul 06 20:53:48.057101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:48.056944    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" event={"ID":"75b46a51-454e-4f33-8229-75cc86f82b75","Type":"ContainerStarted","Data":"21af1c36f789d7db4b9365e469325936a2e99d791a5cc32f8a554949d6fd19a4"}
Jul 06 20:53:48.057101 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:48.057011    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:48.083744 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:48.083688    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" podStartSLOduration=1.545935181 podStartE2EDuration="2.083670891s" totalImagesPullingTime="537.73571ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:53:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:53:48.082877985 +0000 UTC m=+1829.366928525" watchObservedRunningTime="2026-07-06 20:53:48.083670891 +0000 UTC m=+1829.367721431"
Jul 06 20:53:54.067046 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:54.067017    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:58.099001 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.098962    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:58.099435 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.099253    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" podUID="75b46a51-454e-4f33-8229-75cc86f82b75" containerName="maas-api" containerID="cri-o://d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527" gracePeriod=30
Jul 06 20:53:58.348336 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.348312    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:58.462926 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.462837    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") pod \"75b46a51-454e-4f33-8229-75cc86f82b75\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") "
Jul 06 20:53:58.462926 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.462877    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca\") pod \"75b46a51-454e-4f33-8229-75cc86f82b75\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") "
Jul 06 20:53:58.463109 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.462954    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7w7n\" (UniqueName: \"kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n\") pod \"75b46a51-454e-4f33-8229-75cc86f82b75\" (UID: \"75b46a51-454e-4f33-8229-75cc86f82b75\") "
Jul 06 20:53:58.463309 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.463280    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "75b46a51-454e-4f33-8229-75cc86f82b75" (UID: "75b46a51-454e-4f33-8229-75cc86f82b75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:53:58.465063 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.465037    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n" (OuterVolumeSpecName: "kube-api-access-n7w7n") pod "75b46a51-454e-4f33-8229-75cc86f82b75" (UID: "75b46a51-454e-4f33-8229-75cc86f82b75"). InnerVolumeSpecName "kube-api-access-n7w7n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:53:58.465209 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.465194    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "75b46a51-454e-4f33-8229-75cc86f82b75" (UID: "75b46a51-454e-4f33-8229-75cc86f82b75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:53:58.563959 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.563922    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/75b46a51-454e-4f33-8229-75cc86f82b75-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:53:58.563959 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.563953    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/75b46a51-454e-4f33-8229-75cc86f82b75-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:53:58.563959 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:58.563965    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7w7n\" (UniqueName: \"kubernetes.io/projected/75b46a51-454e-4f33-8229-75cc86f82b75-kube-api-access-n7w7n\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:53:59.107713 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.107674    2554 generic.go:356] "Generic (PLEG): container finished" podID="75b46a51-454e-4f33-8229-75cc86f82b75" containerID="d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527" exitCode=0
Jul 06 20:53:59.108122 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.107724    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"
Jul 06 20:53:59.108122 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.107752    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" event={"ID":"75b46a51-454e-4f33-8229-75cc86f82b75","Type":"ContainerDied","Data":"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"}
Jul 06 20:53:59.108122 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.107805    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86" event={"ID":"75b46a51-454e-4f33-8229-75cc86f82b75","Type":"ContainerDied","Data":"21af1c36f789d7db4b9365e469325936a2e99d791a5cc32f8a554949d6fd19a4"}
Jul 06 20:53:59.108122 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.107838    2554 scope.go:117] "RemoveContainer" containerID="d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"
Jul 06 20:53:59.117389 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.117363    2554 scope.go:117] "RemoveContainer" containerID="d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"
Jul 06 20:53:59.117675 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:53:59.117656    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527\": container with ID starting with d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527 not found: ID does not exist" containerID="d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"
Jul 06 20:53:59.117743 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.117686    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527"} err="failed to get container status \"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527\": rpc error: code = NotFound desc = could not find container \"d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527\": container with ID starting with d77c3dd315cda626634fb1be6937f54bc4152d14c3b23e0b2e309114d6e23527 not found: ID does not exist"
Jul 06 20:53:59.129183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.129149    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:59.134030 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.134005    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-58c7cdd8-6b6db9cb4b-wdc86"]
Jul 06 20:53:59.282145 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:53:59.282112    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75b46a51-454e-4f33-8229-75cc86f82b75" path="/var/lib/kubelet/pods/75b46a51-454e-4f33-8229-75cc86f82b75/volumes"
Jul 06 20:54:28.052243 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.052210    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:54:28.052713 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.052616    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75b46a51-454e-4f33-8229-75cc86f82b75" containerName="maas-api"
Jul 06 20:54:28.052713 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.052636    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="75b46a51-454e-4f33-8229-75cc86f82b75" containerName="maas-api"
Jul 06 20:54:28.052786 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.052730    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="75b46a51-454e-4f33-8229-75cc86f82b75" containerName="maas-api"
Jul 06 20:54:28.055614 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.055597    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.058322 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.058298    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-1726c2fe-serving-cert\""
Jul 06 20:54:28.065600 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.065576    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:54:28.098110 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.098069    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tb2q4\" (UniqueName: \"kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.098110 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.098111    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.098309 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.098130    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.199281 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.199238    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tb2q4\" (UniqueName: \"kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.199281 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.199288    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.199490 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.199392    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.200056 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.200035    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.201699 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.201677    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.210421 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.210396    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tb2q4\" (UniqueName: \"kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4\") pod \"maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") " pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.367547 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.367448    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:28.502904 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:28.502875    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:54:28.506105 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:54:28.506069    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc66ea4d_b834_40ff_8e48_c5d3370d66ae.slice/crio-c7783f135932453cc719e8e1873cdd80009f7cba53f0e923bc9e513ab7d384e8 WatchSource:0}: Error finding container c7783f135932453cc719e8e1873cdd80009f7cba53f0e923bc9e513ab7d384e8: Status 404 returned error can't find the container with id c7783f135932453cc719e8e1873cdd80009f7cba53f0e923bc9e513ab7d384e8
Jul 06 20:54:29.245139 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:29.245069    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" event={"ID":"fc66ea4d-b834-40ff-8e48-c5d3370d66ae","Type":"ContainerStarted","Data":"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"}
Jul 06 20:54:29.245139 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:29.245117    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" event={"ID":"fc66ea4d-b834-40ff-8e48-c5d3370d66ae","Type":"ContainerStarted","Data":"c7783f135932453cc719e8e1873cdd80009f7cba53f0e923bc9e513ab7d384e8"}
Jul 06 20:54:29.245557 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:29.245208    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:54:29.262620 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:29.262577    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" podStartSLOduration=0.891143212 podStartE2EDuration="1.262557916s" totalImagesPullingTime="371.414704ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:54:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:54:29.261797616 +0000 UTC m=+1870.545848172" watchObservedRunningTime="2026-07-06 20:54:29.262557916 +0000 UTC m=+1870.546608458"
Jul 06 20:54:35.254844 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:54:35.254814    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:55:10.791408 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:10.791370    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:55:10.791968 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:10.791653    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" podUID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" containerName="maas-api" containerID="cri-o://50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5" gracePeriod=30
Jul 06 20:55:11.043363 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.043300    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:55:11.077217 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.077192    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca\") pod \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") "
Jul 06 20:55:11.077388 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.077234    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls\") pod \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") "
Jul 06 20:55:11.077388 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.077333    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tb2q4\" (UniqueName: \"kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4\") pod \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\" (UID: \"fc66ea4d-b834-40ff-8e48-c5d3370d66ae\") "
Jul 06 20:55:11.077636 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.077606    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc66ea4d-b834-40ff-8e48-c5d3370d66ae" (UID: "fc66ea4d-b834-40ff-8e48-c5d3370d66ae"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:55:11.079583 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.079513    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc66ea4d-b834-40ff-8e48-c5d3370d66ae" (UID: "fc66ea4d-b834-40ff-8e48-c5d3370d66ae"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:55:11.079691 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.079591    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4" (OuterVolumeSpecName: "kube-api-access-tb2q4") pod "fc66ea4d-b834-40ff-8e48-c5d3370d66ae" (UID: "fc66ea4d-b834-40ff-8e48-c5d3370d66ae"). InnerVolumeSpecName "kube-api-access-tb2q4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:55:11.178039 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.177985    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tb2q4\" (UniqueName: \"kubernetes.io/projected/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-kube-api-access-tb2q4\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:55:11.178039 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.178032    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:55:11.178039 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.178043    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc66ea4d-b834-40ff-8e48-c5d3370d66ae-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:55:11.443889 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.443799    2554 generic.go:356] "Generic (PLEG): container finished" podID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" containerID="50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5" exitCode=0
Jul 06 20:55:11.443889 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.443858    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"
Jul 06 20:55:11.443889 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.443876    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" event={"ID":"fc66ea4d-b834-40ff-8e48-c5d3370d66ae","Type":"ContainerDied","Data":"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"}
Jul 06 20:55:11.444173 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.443935    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk" event={"ID":"fc66ea4d-b834-40ff-8e48-c5d3370d66ae","Type":"ContainerDied","Data":"c7783f135932453cc719e8e1873cdd80009f7cba53f0e923bc9e513ab7d384e8"}
Jul 06 20:55:11.444173 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.443965    2554 scope.go:117] "RemoveContainer" containerID="50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"
Jul 06 20:55:11.453845 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.453828    2554 scope.go:117] "RemoveContainer" containerID="50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"
Jul 06 20:55:11.454161 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:55:11.454140    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5\": container with ID starting with 50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5 not found: ID does not exist" containerID="50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"
Jul 06 20:55:11.454220 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.454170    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5"} err="failed to get container status \"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5\": rpc error: code = NotFound desc = could not find container \"50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5\": container with ID starting with 50ad45b1858b32957c20b7c7e6d39f0b590e0b33c50da4b9df4ab1202010f7f5 not found: ID does not exist"
Jul 06 20:55:11.463161 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.463138    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:55:11.468218 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:11.468196    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-1726c2fe-6c66c64db6-4sxtk"]
Jul 06 20:55:13.280928 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:55:13.280902    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" path="/var/lib/kubelet/pods/fc66ea4d-b834-40ff-8e48-c5d3370d66ae/volumes"
Jul 06 20:56:48.022584 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.022545    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:56:48.025025 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.022932    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" containerName="maas-api"
Jul 06 20:56:48.025025 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.022960    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" containerName="maas-api"
Jul 06 20:56:48.025025 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.023059    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc66ea4d-b834-40ff-8e48-c5d3370d66ae" containerName="maas-api"
Jul 06 20:56:48.025800 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.025782    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.057820 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.057788    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-8e767e-serving-cert\""
Jul 06 20:56:48.091630 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.091603    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:56:48.100947 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.100922    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdj9w\" (UniqueName: \"kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.101083 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.100971    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.101083 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.101056    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.202189 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.202136    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.202381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.202222    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.202381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.202267    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdj9w\" (UniqueName: \"kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.202970 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.202949    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.204587 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.204556    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.214375 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.214352    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdj9w\" (UniqueName: \"kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w\") pod \"maas-api-e2e-inf-a-8e767e-549556848d-kjs98\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") " pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.335706 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.335624    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:48.467139 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.467115    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:56:48.469186 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:56:48.469147    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9206763c_6a42_4b78_a982_7cc4cf3a9d4e.slice/crio-8e4a2dbbeb9eec712fdbee3325fa82587a964978909220420d15ae39d249c00a WatchSource:0}: Error finding container 8e4a2dbbeb9eec712fdbee3325fa82587a964978909220420d15ae39d249c00a: Status 404 returned error can't find the container with id 8e4a2dbbeb9eec712fdbee3325fa82587a964978909220420d15ae39d249c00a
Jul 06 20:56:48.470444 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.470427    2554 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 06 20:56:48.875832 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:48.875799    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" event={"ID":"9206763c-6a42-4b78-a982-7cc4cf3a9d4e","Type":"ContainerStarted","Data":"8e4a2dbbeb9eec712fdbee3325fa82587a964978909220420d15ae39d249c00a"}
Jul 06 20:56:49.882605 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:49.882572    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" event={"ID":"9206763c-6a42-4b78-a982-7cc4cf3a9d4e","Type":"ContainerStarted","Data":"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"}
Jul 06 20:56:49.883045 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:49.882659    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:49.900299 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:49.900252    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" podStartSLOduration=2.369855561 podStartE2EDuration="2.900238814s" totalImagesPullingTime="530.383253ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:56:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:56:49.899162339 +0000 UTC m=+2011.183212858" watchObservedRunningTime="2026-07-06 20:56:49.900238814 +0000 UTC m=+2011.184289355"
Jul 06 20:56:55.892607 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:55.892573    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:56:56.103887 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.103851    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:56:56.107338 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.107314    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.113902 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.113878    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-60229a-serving-cert\""
Jul 06 20:56:56.130879 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.130845    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:56:56.276052 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.276013    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.276229 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.276078    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.276229 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.276109    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qzjj8\" (UniqueName: \"kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.377427 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.377382    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.377651 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.377452    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.377651 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.377482    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qzjj8\" (UniqueName: \"kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.377651 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:56:56.377616    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-inf-b-60229a-serving-cert: secret "maas-api-e2e-inf-b-60229a-serving-cert" not found
Jul 06 20:56:56.377824 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:56:56.377693    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls podName:1816cc8c-fb92-4477-acd8-e3be97d7e259 nodeName:}" failed. No retries permitted until 2026-07-06 20:56:56.877672945 +0000 UTC m=+2018.161723465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls") pod "maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" (UID: "1816cc8c-fb92-4477-acd8-e3be97d7e259") : secret "maas-api-e2e-inf-b-60229a-serving-cert" not found
Jul 06 20:56:56.378111 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.378090    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.394234 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.394199    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qzjj8\" (UniqueName: \"kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.882830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.882792    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:56.885227 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:56.885194    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") pod \"maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") " pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:57.018880 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.018838    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:57.162292 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.162264    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:56:57.164840 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:56:57.164810    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1816cc8c_fb92_4477_acd8_e3be97d7e259.slice/crio-7a25681027362b9e0e8ce847a38de61e4aedcd9f601d6c2ae3898e52d180a161 WatchSource:0}: Error finding container 7a25681027362b9e0e8ce847a38de61e4aedcd9f601d6c2ae3898e52d180a161: Status 404 returned error can't find the container with id 7a25681027362b9e0e8ce847a38de61e4aedcd9f601d6c2ae3898e52d180a161
Jul 06 20:56:57.921252 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.921216    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" event={"ID":"1816cc8c-fb92-4477-acd8-e3be97d7e259","Type":"ContainerStarted","Data":"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"}
Jul 06 20:56:57.921464 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.921300    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" event={"ID":"1816cc8c-fb92-4477-acd8-e3be97d7e259","Type":"ContainerStarted","Data":"7a25681027362b9e0e8ce847a38de61e4aedcd9f601d6c2ae3898e52d180a161"}
Jul 06 20:56:57.921464 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.921329    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:56:57.944830 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:56:57.944765    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" podStartSLOduration=1.348029275 podStartE2EDuration="1.944746799s" totalImagesPullingTime="596.717524ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:56:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:56:57.942782954 +0000 UTC m=+2019.226833603" watchObservedRunningTime="2026-07-06 20:56:57.944746799 +0000 UTC m=+2019.228797393"
Jul 06 20:57:03.932398 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:57:03.932370    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:58:06.354381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.354337    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:58:06.354979 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.354611    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" podUID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" containerName="maas-api" containerID="cri-o://d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451" gracePeriod=30
Jul 06 20:58:06.604850 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.604786    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:58:06.780156 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.780113    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca\") pod \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") "
Jul 06 20:58:06.780326 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.780194    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdj9w\" (UniqueName: \"kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w\") pod \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") "
Jul 06 20:58:06.780326 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.780246    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls\") pod \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\" (UID: \"9206763c-6a42-4b78-a982-7cc4cf3a9d4e\") "
Jul 06 20:58:06.780517 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.780491    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9206763c-6a42-4b78-a982-7cc4cf3a9d4e" (UID: "9206763c-6a42-4b78-a982-7cc4cf3a9d4e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:58:06.782305 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.782276    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9206763c-6a42-4b78-a982-7cc4cf3a9d4e" (UID: "9206763c-6a42-4b78-a982-7cc4cf3a9d4e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:58:06.782412 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.782308    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w" (OuterVolumeSpecName: "kube-api-access-tdj9w") pod "9206763c-6a42-4b78-a982-7cc4cf3a9d4e" (UID: "9206763c-6a42-4b78-a982-7cc4cf3a9d4e"). InnerVolumeSpecName "kube-api-access-tdj9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:58:06.881571 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.881452    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdj9w\" (UniqueName: \"kubernetes.io/projected/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-kube-api-access-tdj9w\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:06.881571 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.881485    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:06.881571 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:06.881494    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9206763c-6a42-4b78-a982-7cc4cf3a9d4e-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:07.237053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.236970    2554 generic.go:356] "Generic (PLEG): container finished" podID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" containerID="d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451" exitCode=0
Jul 06 20:58:07.237053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.237029    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"
Jul 06 20:58:07.237053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.237042    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" event={"ID":"9206763c-6a42-4b78-a982-7cc4cf3a9d4e","Type":"ContainerDied","Data":"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"}
Jul 06 20:58:07.237262 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.237113    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98" event={"ID":"9206763c-6a42-4b78-a982-7cc4cf3a9d4e","Type":"ContainerDied","Data":"8e4a2dbbeb9eec712fdbee3325fa82587a964978909220420d15ae39d249c00a"}
Jul 06 20:58:07.237262 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.237135    2554 scope.go:117] "RemoveContainer" containerID="d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"
Jul 06 20:58:07.247833 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.247807    2554 scope.go:117] "RemoveContainer" containerID="d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"
Jul 06 20:58:07.248611 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:58:07.248569    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451\": container with ID starting with d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451 not found: ID does not exist" containerID="d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"
Jul 06 20:58:07.248709 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.248618    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451"} err="failed to get container status \"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451\": rpc error: code = NotFound desc = could not find container \"d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451\": container with ID starting with d5ca9197a0f1b586f68346f0ec4706538c334bb3cad88ab38bf5bcea253a2451 not found: ID does not exist"
Jul 06 20:58:07.262717 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.262690    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:58:07.269485 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.269460    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-8e767e-549556848d-kjs98"]
Jul 06 20:58:07.281941 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:07.281919    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" path="/var/lib/kubelet/pods/9206763c-6a42-4b78-a982-7cc4cf3a9d4e/volumes"
Jul 06 20:58:19.267384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:19.267345    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:58:19.269696 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:19.269674    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 20:58:49.863407 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:49.863372    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:58:49.863933 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:49.863644    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" podUID="1816cc8c-fb92-4477-acd8-e3be97d7e259" containerName="maas-api" containerID="cri-o://446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091" gracePeriod=30
Jul 06 20:58:50.110881 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.110853    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:58:50.214950 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.214875    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca\") pod \"1816cc8c-fb92-4477-acd8-e3be97d7e259\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") "
Jul 06 20:58:50.215093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.214965    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") pod \"1816cc8c-fb92-4477-acd8-e3be97d7e259\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") "
Jul 06 20:58:50.215093 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.215008    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qzjj8\" (UniqueName: \"kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8\") pod \"1816cc8c-fb92-4477-acd8-e3be97d7e259\" (UID: \"1816cc8c-fb92-4477-acd8-e3be97d7e259\") "
Jul 06 20:58:50.215384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.215195    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1816cc8c-fb92-4477-acd8-e3be97d7e259" (UID: "1816cc8c-fb92-4477-acd8-e3be97d7e259"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:58:50.215384 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.215297    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1816cc8c-fb92-4477-acd8-e3be97d7e259-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:50.217044 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.217020    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1816cc8c-fb92-4477-acd8-e3be97d7e259" (UID: "1816cc8c-fb92-4477-acd8-e3be97d7e259"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:58:50.217126 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.217098    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8" (OuterVolumeSpecName: "kube-api-access-qzjj8") pod "1816cc8c-fb92-4477-acd8-e3be97d7e259" (UID: "1816cc8c-fb92-4477-acd8-e3be97d7e259"). InnerVolumeSpecName "kube-api-access-qzjj8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:58:50.315882 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.315848    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1816cc8c-fb92-4477-acd8-e3be97d7e259-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:50.315882 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.315874    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qzjj8\" (UniqueName: \"kubernetes.io/projected/1816cc8c-fb92-4477-acd8-e3be97d7e259-kube-api-access-qzjj8\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:58:50.439545 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.439504    2554 generic.go:356] "Generic (PLEG): container finished" podID="1816cc8c-fb92-4477-acd8-e3be97d7e259" containerID="446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091" exitCode=0
Jul 06 20:58:50.439742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.439578    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"
Jul 06 20:58:50.439742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.439599    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" event={"ID":"1816cc8c-fb92-4477-acd8-e3be97d7e259","Type":"ContainerDied","Data":"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"}
Jul 06 20:58:50.439742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.439662    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg" event={"ID":"1816cc8c-fb92-4477-acd8-e3be97d7e259","Type":"ContainerDied","Data":"7a25681027362b9e0e8ce847a38de61e4aedcd9f601d6c2ae3898e52d180a161"}
Jul 06 20:58:50.439742 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.439695    2554 scope.go:117] "RemoveContainer" containerID="446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"
Jul 06 20:58:50.451971 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.451949    2554 scope.go:117] "RemoveContainer" containerID="446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"
Jul 06 20:58:50.452351 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:58:50.452245    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091\": container with ID starting with 446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091 not found: ID does not exist" containerID="446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"
Jul 06 20:58:50.452351 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.452286    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091"} err="failed to get container status \"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091\": rpc error: code = NotFound desc = could not find container \"446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091\": container with ID starting with 446f2a898860b8168557c7fe94b54e2813d43a9504154d8229f656a94b3b1091 not found: ID does not exist"
Jul 06 20:58:50.466493 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.466440    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:58:50.472369 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:50.472348    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-60229a-7864657dcf-7qbmg"]
Jul 06 20:58:51.281396 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:58:51.281365    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1816cc8c-fb92-4477-acd8-e3be97d7e259" path="/var/lib/kubelet/pods/1816cc8c-fb92-4477-acd8-e3be97d7e259/volumes"
Jul 06 20:59:00.027453 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027419    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027766    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" containerName="maas-api"
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027780    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" containerName="maas-api"
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027790    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1816cc8c-fb92-4477-acd8-e3be97d7e259" containerName="maas-api"
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027796    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1816cc8c-fb92-4477-acd8-e3be97d7e259" containerName="maas-api"
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027868    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1816cc8c-fb92-4477-acd8-e3be97d7e259" containerName="maas-api"
Jul 06 20:59:00.027906 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.027879    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="9206763c-6a42-4b78-a982-7cc4cf3a9d4e" containerName="maas-api"
Jul 06 20:59:00.030769 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.030750    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.040190 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.040166    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-c6b4b3-serving-cert\""
Jul 06 20:59:00.055419 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.055391    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:00.097814 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.097775    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.097985 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.097827    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxdcb\" (UniqueName: \"kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.097985 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.097929    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.199344 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.199310    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.199360    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.199616 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.199415    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxdcb\" (UniqueName: \"kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.199616 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:59:00.199469    2554 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-c6b4b3-serving-cert: secret "maas-api-e2e-api-a-c6b4b3-serving-cert" not found
Jul 06 20:59:00.199616 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:59:00.199565    2554 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls podName:d9cf75fa-0599-4c98-8a86-464b55a9e6d4 nodeName:}" failed. No retries permitted until 2026-07-06 20:59:00.699515895 +0000 UTC m=+2141.983566423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls") pod "maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" (UID: "d9cf75fa-0599-4c98-8a86-464b55a9e6d4") : secret "maas-api-e2e-api-a-c6b4b3-serving-cert" not found
Jul 06 20:59:00.200115 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.200092    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.220396 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.220370    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxdcb\" (UniqueName: \"kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.704265 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.704233    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.706714 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.706694    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") pod \"maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") " pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:00.940915 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:00.940881    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:01.087310 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:01.087286    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:01.089214 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:59:01.089186    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9cf75fa_0599_4c98_8a86_464b55a9e6d4.slice/crio-0a784ca7d6da33cd7df47d5b800d3c96e2331668626bf256fd47b2ed5b4453d1 WatchSource:0}: Error finding container 0a784ca7d6da33cd7df47d5b800d3c96e2331668626bf256fd47b2ed5b4453d1: Status 404 returned error can't find the container with id 0a784ca7d6da33cd7df47d5b800d3c96e2331668626bf256fd47b2ed5b4453d1
Jul 06 20:59:01.488543 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:01.488503    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" event={"ID":"d9cf75fa-0599-4c98-8a86-464b55a9e6d4","Type":"ContainerStarted","Data":"0a784ca7d6da33cd7df47d5b800d3c96e2331668626bf256fd47b2ed5b4453d1"}
Jul 06 20:59:02.494790 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:02.494753    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" event={"ID":"d9cf75fa-0599-4c98-8a86-464b55a9e6d4","Type":"ContainerStarted","Data":"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"}
Jul 06 20:59:02.495249 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:02.494828    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:02.520041 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:02.514815    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" podStartSLOduration=2.939819807 podStartE2EDuration="3.514794504s" totalImagesPullingTime="574.974697ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:02.512717651 +0000 UTC m=+2143.796768191" watchObservedRunningTime="2026-07-06 20:59:02.514794504 +0000 UTC m=+2143.798845046"
Jul 06 20:59:08.046383 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.046348    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:08.050243 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.050215    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.053200 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.053170    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-275fd7-serving-cert\""
Jul 06 20:59:08.062108 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.062081    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:08.169100 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.169066    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.169275 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.169131    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67nkc\" (UniqueName: \"kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.169275 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.169190    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.270660 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.270625    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67nkc\" (UniqueName: \"kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.270877 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.270687    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.270877 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.270717    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.271309 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.271276    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.273150 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.273129    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.289279 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.289257    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67nkc\" (UniqueName: \"kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc\") pod \"maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") " pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.361975 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.361892    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:08.504746 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.504724    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:08.506917 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.506893    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:08.507665 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:59:08.507641    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef136b36_8f39_43f4_8055_1a8ebbe4e7a2.slice/crio-07b655e704bad0291e03e1ea5e779ee9c34c60c94de6cac2fa914e9c195aec1d WatchSource:0}: Error finding container 07b655e704bad0291e03e1ea5e779ee9c34c60c94de6cac2fa914e9c195aec1d: Status 404 returned error can't find the container with id 07b655e704bad0291e03e1ea5e779ee9c34c60c94de6cac2fa914e9c195aec1d
Jul 06 20:59:08.522961 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:08.522933    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" event={"ID":"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2","Type":"ContainerStarted","Data":"07b655e704bad0291e03e1ea5e779ee9c34c60c94de6cac2fa914e9c195aec1d"}
Jul 06 20:59:09.530474 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:09.530441    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" event={"ID":"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2","Type":"ContainerStarted","Data":"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"}
Jul 06 20:59:09.530843 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:09.530511    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:09.551424 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:09.551379    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" podStartSLOduration=1.126572071 podStartE2EDuration="1.55136323s" totalImagesPullingTime="424.791159ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:59:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:09.549815335 +0000 UTC m=+2150.833865875" watchObservedRunningTime="2026-07-06 20:59:09.55136323 +0000 UTC m=+2150.835413775"
Jul 06 20:59:15.540298 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:15.540222    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:21.603690 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.603657    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:21.604167 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.603871    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" podUID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" containerName="maas-api" containerID="cri-o://bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2" gracePeriod=30
Jul 06 20:59:21.855206 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.855135    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:21.886818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.886782    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") pod \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") "
Jul 06 20:59:21.887000 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.886840    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxdcb\" (UniqueName: \"kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb\") pod \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") "
Jul 06 20:59:21.887000 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.886911    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca\") pod \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\" (UID: \"d9cf75fa-0599-4c98-8a86-464b55a9e6d4\") "
Jul 06 20:59:21.887254 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.887225    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d9cf75fa-0599-4c98-8a86-464b55a9e6d4" (UID: "d9cf75fa-0599-4c98-8a86-464b55a9e6d4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:59:21.888942 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.888918    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb" (OuterVolumeSpecName: "kube-api-access-kxdcb") pod "d9cf75fa-0599-4c98-8a86-464b55a9e6d4" (UID: "d9cf75fa-0599-4c98-8a86-464b55a9e6d4"). InnerVolumeSpecName "kube-api-access-kxdcb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:59:21.889053 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.889036    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d9cf75fa-0599-4c98-8a86-464b55a9e6d4" (UID: "d9cf75fa-0599-4c98-8a86-464b55a9e6d4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:59:21.988428 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.988389    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:21.988428 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.988422    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxdcb\" (UniqueName: \"kubernetes.io/projected/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-kube-api-access-kxdcb\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:21.988428 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:21.988432    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d9cf75fa-0599-4c98-8a86-464b55a9e6d4-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:22.591721 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.591680    2554 generic.go:356] "Generic (PLEG): container finished" podID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" containerID="bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2" exitCode=0
Jul 06 20:59:22.591721 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.591725    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"
Jul 06 20:59:22.592001 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.591767    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" event={"ID":"d9cf75fa-0599-4c98-8a86-464b55a9e6d4","Type":"ContainerDied","Data":"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"}
Jul 06 20:59:22.592001 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.591833    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl" event={"ID":"d9cf75fa-0599-4c98-8a86-464b55a9e6d4","Type":"ContainerDied","Data":"0a784ca7d6da33cd7df47d5b800d3c96e2331668626bf256fd47b2ed5b4453d1"}
Jul 06 20:59:22.592001 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.591862    2554 scope.go:117] "RemoveContainer" containerID="bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"
Jul 06 20:59:22.601058 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.601038    2554 scope.go:117] "RemoveContainer" containerID="bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"
Jul 06 20:59:22.601370 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:59:22.601353    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2\": container with ID starting with bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2 not found: ID does not exist" containerID="bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"
Jul 06 20:59:22.601436 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.601378    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2"} err="failed to get container status \"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2\": rpc error: code = NotFound desc = could not find container \"bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2\": container with ID starting with bc3964b0fc23452fde61a607bc11750767311a04b98b15f7233c71a1f47439e2 not found: ID does not exist"
Jul 06 20:59:22.613276 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.613240    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:22.620239 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:22.620209    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-c6b4b3-b8967ffb5-d5mgl"]
Jul 06 20:59:23.281909 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:23.281875    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" path="/var/lib/kubelet/pods/d9cf75fa-0599-4c98-8a86-464b55a9e6d4/volumes"
Jul 06 20:59:29.143974 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.143939    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:29.144406 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.144166    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" podUID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" containerName="maas-api" containerID="cri-o://5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a" gracePeriod=30
Jul 06 20:59:29.398654 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.398557    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:29.453960 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.453924    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca\") pod \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") "
Jul 06 20:59:29.454183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.453967    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-67nkc\" (UniqueName: \"kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc\") pod \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") "
Jul 06 20:59:29.454183 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.454099    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls\") pod \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\" (UID: \"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2\") "
Jul 06 20:59:29.454349 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.454312    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" (UID: "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 20:59:29.456155 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.456130    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc" (OuterVolumeSpecName: "kube-api-access-67nkc") pod "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" (UID: "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2"). InnerVolumeSpecName "kube-api-access-67nkc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 20:59:29.456268 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.456198    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" (UID: "ef136b36-8f39-43f4-8055-1a8ebbe4e7a2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 20:59:29.555753 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.555713    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:29.555753 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.555745    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:29.555753 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.555755    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67nkc\" (UniqueName: \"kubernetes.io/projected/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2-kube-api-access-67nkc\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 20:59:29.625170 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.625133    2554 generic.go:356] "Generic (PLEG): container finished" podID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" containerID="5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a" exitCode=0
Jul 06 20:59:29.625333 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.625182    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"
Jul 06 20:59:29.625333 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.625222    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" event={"ID":"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2","Type":"ContainerDied","Data":"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"}
Jul 06 20:59:29.625333 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.625276    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz" event={"ID":"ef136b36-8f39-43f4-8055-1a8ebbe4e7a2","Type":"ContainerDied","Data":"07b655e704bad0291e03e1ea5e779ee9c34c60c94de6cac2fa914e9c195aec1d"}
Jul 06 20:59:29.625333 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.625302    2554 scope.go:117] "RemoveContainer" containerID="5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"
Jul 06 20:59:29.635454 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.635434    2554 scope.go:117] "RemoveContainer" containerID="5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"
Jul 06 20:59:29.635791 ip-10-0-128-47 kubenswrapper[2554]: E0706 20:59:29.635772    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a\": container with ID starting with 5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a not found: ID does not exist" containerID="5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"
Jul 06 20:59:29.635872 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.635800    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a"} err="failed to get container status \"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a\": rpc error: code = NotFound desc = could not find container \"5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a\": container with ID starting with 5952c6bcc94233fddfbaba8f1282f594a9a8c9425b3d146b67515e16b111841a not found: ID does not exist"
Jul 06 20:59:29.648525 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.648484    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:29.655745 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:29.655672    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-275fd7-578f5f7ccf-96ljz"]
Jul 06 20:59:31.281237 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:31.281206    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" path="/var/lib/kubelet/pods/ef136b36-8f39-43f4-8055-1a8ebbe4e7a2/volumes"
Jul 06 20:59:39.190858 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.190823    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 20:59:39.191339 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191318    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" containerName="maas-api"
Jul 06 20:59:39.191339 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191337    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" containerName="maas-api"
Jul 06 20:59:39.191463 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191367    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" containerName="maas-api"
Jul 06 20:59:39.191463 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191376    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" containerName="maas-api"
Jul 06 20:59:39.191596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191491    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="ef136b36-8f39-43f4-8055-1a8ebbe4e7a2" containerName="maas-api"
Jul 06 20:59:39.191596 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.191509    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9cf75fa-0599-4c98-8a86-464b55a9e6d4" containerName="maas-api"
Jul 06 20:59:39.195729 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.195706    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.199090 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.199067    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-544d30-serving-cert\""
Jul 06 20:59:39.233950 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.233922    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 20:59:39.243216 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.243183    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.243374 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.243250    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.243374 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.243274    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69cmr\" (UniqueName: \"kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.344049 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.344015    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.344219 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.344057    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69cmr\" (UniqueName: \"kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.344219 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.344105    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.344839 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.344817    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.346601 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.346574    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.367686 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.367660    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-69cmr\" (UniqueName: \"kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr\") pod \"maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") " pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.505408 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.505320    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:39.692374 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:39.692347    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 20:59:39.695234 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:59:39.695207    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc12f1a37_e6db_4646_a52d_7b955232c7e7.slice/crio-54503c28ded7596f403b11062322f5019b792382da7c28240067a152080d2681 WatchSource:0}: Error finding container 54503c28ded7596f403b11062322f5019b792382da7c28240067a152080d2681: Status 404 returned error can't find the container with id 54503c28ded7596f403b11062322f5019b792382da7c28240067a152080d2681
Jul 06 20:59:40.674365 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:40.674334    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" event={"ID":"c12f1a37-e6db-4646-a52d-7b955232c7e7","Type":"ContainerStarted","Data":"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"}
Jul 06 20:59:40.674781 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:40.674384    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" event={"ID":"c12f1a37-e6db-4646-a52d-7b955232c7e7","Type":"ContainerStarted","Data":"54503c28ded7596f403b11062322f5019b792382da7c28240067a152080d2681"}
Jul 06 20:59:40.674781 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:40.674427    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:40.693318 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:40.693268    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" podStartSLOduration=1.36212121 podStartE2EDuration="1.693250136s" totalImagesPullingTime="331.128926ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:59:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:40.692272505 +0000 UTC m=+2181.976323082" watchObservedRunningTime="2026-07-06 20:59:40.693250136 +0000 UTC m=+2181.977300678"
Jul 06 20:59:46.685037 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:46.685010    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 20:59:47.237336 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.237305    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 20:59:47.241024 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.241005    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.244930 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.244909    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-cd7c86-serving-cert\""
Jul 06 20:59:47.299134 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.299095    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 20:59:47.328589 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.328559    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.328812 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.328607    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vh9t8\" (UniqueName: \"kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.328812 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.328682    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.429279 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.429242    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.429457 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.429293    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vh9t8\" (UniqueName: \"kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.429457 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.429338    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.429888 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.429869    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.431800 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.431775    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.460978 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.460950    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vh9t8\" (UniqueName: \"kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8\") pod \"maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") " pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.551946 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.551910    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:47.732404 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:47.732377    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 20:59:47.735262 ip-10-0-128-47 kubenswrapper[2554]: W0706 20:59:47.735229    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod273abb3b_e78b_4148_8460_502a394cfd5a.slice/crio-98c4511959641631b6ea00cb919c10b706c3ce40ab4364f5f09ad0759abf969f WatchSource:0}: Error finding container 98c4511959641631b6ea00cb919c10b706c3ce40ab4364f5f09ad0759abf969f: Status 404 returned error can't find the container with id 98c4511959641631b6ea00cb919c10b706c3ce40ab4364f5f09ad0759abf969f
Jul 06 20:59:48.711181 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:48.711139    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" event={"ID":"273abb3b-e78b-4148-8460-502a394cfd5a","Type":"ContainerStarted","Data":"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"}
Jul 06 20:59:48.711381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:48.711198    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" event={"ID":"273abb3b-e78b-4148-8460-502a394cfd5a","Type":"ContainerStarted","Data":"98c4511959641631b6ea00cb919c10b706c3ce40ab4364f5f09ad0759abf969f"}
Jul 06 20:59:48.711381 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:48.711283    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 20:59:48.740818 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:48.740758    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" podStartSLOduration=1.454727379 podStartE2EDuration="1.74073852s" totalImagesPullingTime="286.011141ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 20:59:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 20:59:48.74042349 +0000 UTC m=+2190.024474033" watchObservedRunningTime="2026-07-06 20:59:48.74073852 +0000 UTC m=+2190.024789062"
Jul 06 20:59:54.721873 ip-10-0-128-47 kubenswrapper[2554]: I0706 20:59:54.721787    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 21:00:00.156665 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.156628    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"]
Jul 06 21:00:00.160337 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.160315    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:00.160625 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.160604    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29722860-9w262"]
Jul 06 21:00:00.163919 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.163901    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"]
Jul 06 21:00:00.164046 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.164024    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:00.166904 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.166887    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:00.184111 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.184088    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"]
Jul 06 21:00:00.207345 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.207311    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722860-9w262"]
Jul 06 21:00:00.209669 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.209644    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"]
Jul 06 21:00:00.244019 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.243990    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpshw\" (UniqueName: \"kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw\") pod \"maas-api-key-cleanup-29722860-9w262\" (UID: \"0388722a-1b7c-4bd0-8afe-6a5826ab4c55\") " pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:00.244194 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.244042    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk4ww\" (UniqueName: \"kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww\") pod \"maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf\" (UID: \"c2cd8754-793c-4f76-8436-7cdd4d7766ce\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:00.244194 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.244128    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzk5f\" (UniqueName: \"kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f\") pod \"maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw\" (UID: \"5b9787b5-14bd-4142-8998-3153117a7c4e\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:00.345065 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.345028    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpshw\" (UniqueName: \"kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw\") pod \"maas-api-key-cleanup-29722860-9w262\" (UID: \"0388722a-1b7c-4bd0-8afe-6a5826ab4c55\") " pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:00.345065 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.345070    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk4ww\" (UniqueName: \"kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww\") pod \"maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf\" (UID: \"c2cd8754-793c-4f76-8436-7cdd4d7766ce\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:00.345292 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.345111    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzk5f\" (UniqueName: \"kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f\") pod \"maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw\" (UID: \"5b9787b5-14bd-4142-8998-3153117a7c4e\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:00.362299 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.362266    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpshw\" (UniqueName: \"kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw\") pod \"maas-api-key-cleanup-29722860-9w262\" (UID: \"0388722a-1b7c-4bd0-8afe-6a5826ab4c55\") " pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:00.362872 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.362854    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk4ww\" (UniqueName: \"kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww\") pod \"maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf\" (UID: \"c2cd8754-793c-4f76-8436-7cdd4d7766ce\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:00.362951 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.362854    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzk5f\" (UniqueName: \"kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f\") pod \"maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw\" (UID: \"5b9787b5-14bd-4142-8998-3153117a7c4e\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:00.472012 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.471925    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:00.479078 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.479058    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:00.484962 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.484936    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:00.652713 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.652689    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"]
Jul 06 21:00:00.654752 ip-10-0-128-47 kubenswrapper[2554]: W0706 21:00:00.654716    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2cd8754_793c_4f76_8436_7cdd4d7766ce.slice/crio-de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5 WatchSource:0}: Error finding container de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5: Status 404 returned error can't find the container with id de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5
Jul 06 21:00:00.773525 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.773488    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf" event={"ID":"c2cd8754-793c-4f76-8436-7cdd4d7766ce","Type":"ContainerStarted","Data":"de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5"}
Jul 06 21:00:00.891294 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.891200    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"]
Jul 06 21:00:00.894042 ip-10-0-128-47 kubenswrapper[2554]: W0706 21:00:00.894013    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9787b5_14bd_4142_8998_3153117a7c4e.slice/crio-26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd WatchSource:0}: Error finding container 26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd: Status 404 returned error can't find the container with id 26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd
Jul 06 21:00:00.899454 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:00.899431    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29722860-9w262"]
Jul 06 21:00:00.900759 ip-10-0-128-47 kubenswrapper[2554]: W0706 21:00:00.900733    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0388722a_1b7c_4bd0_8afe_6a5826ab4c55.slice/crio-9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b WatchSource:0}: Error finding container 9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b: Status 404 returned error can't find the container with id 9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b
Jul 06 21:00:01.780268 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.780217    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722860-9w262" event={"ID":"0388722a-1b7c-4bd0-8afe-6a5826ab4c55","Type":"ContainerStarted","Data":"9ab35bc58710c528b0b945a82a313c6e8e420e5c46d4e60b4b236dffee96922d"}
Jul 06 21:00:01.780735 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.780286    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722860-9w262" event={"ID":"0388722a-1b7c-4bd0-8afe-6a5826ab4c55","Type":"ContainerStarted","Data":"9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b"}
Jul 06 21:00:01.781809 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.781787    2554 generic.go:356] "Generic (PLEG): container finished" podID="c2cd8754-793c-4f76-8436-7cdd4d7766ce" containerID="43a27d388c5c8cf0e775afd2d4e7d435c653187bea05473bda221c9fba9b4b71" exitCode=0
Jul 06 21:00:01.781898 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.781857    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf" event={"ID":"c2cd8754-793c-4f76-8436-7cdd4d7766ce","Type":"ContainerDied","Data":"43a27d388c5c8cf0e775afd2d4e7d435c653187bea05473bda221c9fba9b4b71"}
Jul 06 21:00:01.783203 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.783178    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw" event={"ID":"5b9787b5-14bd-4142-8998-3153117a7c4e","Type":"ContainerStarted","Data":"26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b"}
Jul 06 21:00:01.783334 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.783228    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw" event={"ID":"5b9787b5-14bd-4142-8998-3153117a7c4e","Type":"ContainerStarted","Data":"26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd"}
Jul 06 21:00:01.809110 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.809058    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29722860-9w262" podStartSLOduration=1.809041795 podStartE2EDuration="1.809041795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:00:01.807335891 +0000 UTC m=+2203.091386434" watchObservedRunningTime="2026-07-06 21:00:01.809041795 +0000 UTC m=+2203.093092335"
Jul 06 21:00:01.866317 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:01.866267    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw" podStartSLOduration=1.866252196 podStartE2EDuration="1.866252196s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:00:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:00:01.865175713 +0000 UTC m=+2203.149226255" watchObservedRunningTime="2026-07-06 21:00:01.866252196 +0000 UTC m=+2203.150302739"
Jul 06 21:00:02.257109 ip-10-0-128-47 kubenswrapper[2554]: E0706 21:00:02.257071    2554 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9787b5_14bd_4142_8998_3153117a7c4e.slice/crio-26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b.scope\": RecentStats: unable to find data in memory cache]"
Jul 06 21:00:02.257286 ip-10-0-128-47 kubenswrapper[2554]: E0706 21:00:02.257189    2554 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b9787b5_14bd_4142_8998_3153117a7c4e.slice/crio-26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0388722a_1b7c_4bd0_8afe_6a5826ab4c55.slice/crio-9ab35bc58710c528b0b945a82a313c6e8e420e5c46d4e60b4b236dffee96922d.scope\": RecentStats: unable to find data in memory cache]"
Jul 06 21:00:02.789227 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.789186    2554 generic.go:356] "Generic (PLEG): container finished" podID="0388722a-1b7c-4bd0-8afe-6a5826ab4c55" containerID="9ab35bc58710c528b0b945a82a313c6e8e420e5c46d4e60b4b236dffee96922d" exitCode=0
Jul 06 21:00:02.789671 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.789277    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722860-9w262" event={"ID":"0388722a-1b7c-4bd0-8afe-6a5826ab4c55","Type":"ContainerDied","Data":"9ab35bc58710c528b0b945a82a313c6e8e420e5c46d4e60b4b236dffee96922d"}
Jul 06 21:00:02.793587 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.793548    2554 generic.go:356] "Generic (PLEG): container finished" podID="5b9787b5-14bd-4142-8998-3153117a7c4e" containerID="26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b" exitCode=0
Jul 06 21:00:02.793762 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.793591    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw" event={"ID":"5b9787b5-14bd-4142-8998-3153117a7c4e","Type":"ContainerDied","Data":"26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b"}
Jul 06 21:00:02.940496 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.940471    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:02.969099 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.969070    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk4ww\" (UniqueName: \"kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww\") pod \"c2cd8754-793c-4f76-8436-7cdd4d7766ce\" (UID: \"c2cd8754-793c-4f76-8436-7cdd4d7766ce\") "
Jul 06 21:00:02.971138 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:02.971112    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww" (OuterVolumeSpecName: "kube-api-access-rk4ww") pod "c2cd8754-793c-4f76-8436-7cdd4d7766ce" (UID: "c2cd8754-793c-4f76-8436-7cdd4d7766ce"). InnerVolumeSpecName "kube-api-access-rk4ww". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:00:03.069789 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.069710    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk4ww\" (UniqueName: \"kubernetes.io/projected/c2cd8754-793c-4f76-8436-7cdd4d7766ce-kube-api-access-rk4ww\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:00:03.799713 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.799685    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"
Jul 06 21:00:03.800103 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.799726    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf" event={"ID":"c2cd8754-793c-4f76-8436-7cdd4d7766ce","Type":"ContainerDied","Data":"de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5"}
Jul 06 21:00:03.800103 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.799772    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="de9685cab53e2ac13f4b64805c0437554b2f03f08f3d5dbb6c4a7a820fa584e5"
Jul 06 21:00:03.960001 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.959979    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:03.972833 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.972809    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:03.977275 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.977255    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vzk5f\" (UniqueName: \"kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f\") pod \"5b9787b5-14bd-4142-8998-3153117a7c4e\" (UID: \"5b9787b5-14bd-4142-8998-3153117a7c4e\") "
Jul 06 21:00:03.979188 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:03.979160    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f" (OuterVolumeSpecName: "kube-api-access-vzk5f") pod "5b9787b5-14bd-4142-8998-3153117a7c4e" (UID: "5b9787b5-14bd-4142-8998-3153117a7c4e"). InnerVolumeSpecName "kube-api-access-vzk5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:00:04.078398 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.078308    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpshw\" (UniqueName: \"kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw\") pod \"0388722a-1b7c-4bd0-8afe-6a5826ab4c55\" (UID: \"0388722a-1b7c-4bd0-8afe-6a5826ab4c55\") "
Jul 06 21:00:04.078630 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.078613    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vzk5f\" (UniqueName: \"kubernetes.io/projected/5b9787b5-14bd-4142-8998-3153117a7c4e-kube-api-access-vzk5f\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:00:04.080424 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.080392    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw" (OuterVolumeSpecName: "kube-api-access-gpshw") pod "0388722a-1b7c-4bd0-8afe-6a5826ab4c55" (UID: "0388722a-1b7c-4bd0-8afe-6a5826ab4c55"). InnerVolumeSpecName "kube-api-access-gpshw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:00:04.179630 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.179593    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpshw\" (UniqueName: \"kubernetes.io/projected/0388722a-1b7c-4bd0-8afe-6a5826ab4c55-kube-api-access-gpshw\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:00:04.805146 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.805114    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29722860-9w262"
Jul 06 21:00:04.805658 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.805155    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29722860-9w262" event={"ID":"0388722a-1b7c-4bd0-8afe-6a5826ab4c55","Type":"ContainerDied","Data":"9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b"}
Jul 06 21:00:04.805658 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.805206    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9fc40f808910ab1ea854b0a8f9df704c26645b3bc030c3ab74708d131fa4477b"
Jul 06 21:00:04.806410 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.806387    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"
Jul 06 21:00:04.806514 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.806411    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw" event={"ID":"5b9787b5-14bd-4142-8998-3153117a7c4e","Type":"ContainerDied","Data":"26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd"}
Jul 06 21:00:04.806514 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:00:04.806449    2554 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="26b25d8d6c8891ed481a48654ec3dd326b3331554349095ba984734bea6330bd"
Jul 06 21:02:33.114547 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114462    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114807    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0388722a-1b7c-4bd0-8afe-6a5826ab4c55" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114817    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="0388722a-1b7c-4bd0-8afe-6a5826ab4c55" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114829    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b9787b5-14bd-4142-8998-3153117a7c4e" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114834    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b9787b5-14bd-4142-8998-3153117a7c4e" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114844    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2cd8754-793c-4f76-8436-7cdd4d7766ce" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114850    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2cd8754-793c-4f76-8436-7cdd4d7766ce" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114917    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2cd8754-793c-4f76-8436-7cdd4d7766ce" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114924    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b9787b5-14bd-4142-8998-3153117a7c4e" containerName="cleanup"
Jul 06 21:02:33.114973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.114931    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="0388722a-1b7c-4bd0-8afe-6a5826ab4c55" containerName="cleanup"
Jul 06 21:02:33.117767 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.117751    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.123218 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.123197    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-544d30\"/\"kube-root-ca.crt\""
Jul 06 21:02:33.123360 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.123202    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-544d30\"/\"sub-test-model-544d30-kserve-self-signed-certs\""
Jul 06 21:02:33.123360 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.123238    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-544d30\"/\"default-dockercfg-pjfp2\""
Jul 06 21:02:33.126643 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.126626    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-544d30\"/\"openshift-service-ca.crt\""
Jul 06 21:02:33.150017 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.149995    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:02:33.188346 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188316    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.188458 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188352    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.188458 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188369    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.188458 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188392    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.188458 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188440    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.188624 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.188458    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zstrr\" (UniqueName: \"kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.289836 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289786    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.289836 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289836    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290087 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289879    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290087 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289901    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290087 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289944    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290087 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.289969    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zstrr\" (UniqueName: \"kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290315 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.290299    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290373 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.290307    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.290420 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.290374    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.292218 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.292193    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.292496 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.292475    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.303874 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.303851    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zstrr\" (UniqueName: \"kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr\") pod \"sub-test-model-544d30-kserve-86478ffcd7-qslmw\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") " pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.426867 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.426787    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:33.569178 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:33.569154    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:02:33.571272 ip-10-0-128-47 kubenswrapper[2554]: W0706 21:02:33.571246    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1794830b_6c81_42aa_8227_2042d9425434.slice/crio-103476e0acac6b7546d75ea8863e32bc59e70f4dad6b7655a8fc740bcf767c7c WatchSource:0}: Error finding container 103476e0acac6b7546d75ea8863e32bc59e70f4dad6b7655a8fc740bcf767c7c: Status 404 returned error can't find the container with id 103476e0acac6b7546d75ea8863e32bc59e70f4dad6b7655a8fc740bcf767c7c
Jul 06 21:02:34.494777 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:34.494742    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" event={"ID":"1794830b-6c81-42aa-8227-2042d9425434","Type":"ContainerStarted","Data":"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"}
Jul 06 21:02:34.494777 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:34.494785    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" event={"ID":"1794830b-6c81-42aa-8227-2042d9425434","Type":"ContainerStarted","Data":"103476e0acac6b7546d75ea8863e32bc59e70f4dad6b7655a8fc740bcf767c7c"}
Jul 06 21:02:34.519166 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:34.519107    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" podStartSLOduration=1.519091168 podStartE2EDuration="1.519091168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-06 21:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:02:34.518080368 +0000 UTC m=+2355.802130910" watchObservedRunningTime="2026-07-06 21:02:34.519091168 +0000 UTC m=+2355.803141709"
Jul 06 21:02:43.427242 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:43.427198    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:43.427794 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:43.427259    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:43.440326 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:43.440295    2554 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:02:43.553785 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:02:43.553754    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:03:19.297393 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:19.297368    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 21:03:19.300742 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:19.300724    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-59hxf_36417a94-cd86-478c-9b05-eb89b8746d19/ovn-acl-logging/0.log"
Jul 06 21:03:41.498803 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:41.498728    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:03:56.518465 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.518423    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" podUID="1794830b-6c81-42aa-8227-2042d9425434" containerName="main" containerID="cri-o://82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4" gracePeriod=45
Jul 06 21:03:56.779252 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.779191    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:03:56.827033 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.826997    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827225 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827088    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827225 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827122    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zstrr\" (UniqueName: \"kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827225 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827166    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827440 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827410    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home" (OuterVolumeSpecName: "home") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:56.827502 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827424    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:56.827681 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827641    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827815 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827700    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm\") pod \"1794830b-6c81-42aa-8227-2042d9425434\" (UID: \"1794830b-6c81-42aa-8227-2042d9425434\") "
Jul 06 21:03:56.827984 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827957    2554 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-home\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:56.827984 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.827976    2554 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-tmp-dir\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:56.828160 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.828097    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache" (OuterVolumeSpecName: "model-cache") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:56.829701 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.829678    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr" (OuterVolumeSpecName: "kube-api-access-zstrr") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "kube-api-access-zstrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:03:56.829780 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.829725    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:03:56.830140 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.830122    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm" (OuterVolumeSpecName: "dshm") pod "1794830b-6c81-42aa-8227-2042d9425434" (UID: "1794830b-6c81-42aa-8227-2042d9425434"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 06 21:03:56.887639 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.887601    2554 generic.go:356] "Generic (PLEG): container finished" podID="1794830b-6c81-42aa-8227-2042d9425434" containerID="82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4" exitCode=0
Jul 06 21:03:56.887819 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.887651    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"
Jul 06 21:03:56.887819 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.887661    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" event={"ID":"1794830b-6c81-42aa-8227-2042d9425434","Type":"ContainerDied","Data":"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"}
Jul 06 21:03:56.887819 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.887692    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw" event={"ID":"1794830b-6c81-42aa-8227-2042d9425434","Type":"ContainerDied","Data":"103476e0acac6b7546d75ea8863e32bc59e70f4dad6b7655a8fc740bcf767c7c"}
Jul 06 21:03:56.887819 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.887708    2554 scope.go:117] "RemoveContainer" containerID="82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"
Jul 06 21:03:56.896734 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.896713    2554 scope.go:117] "RemoveContainer" containerID="82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"
Jul 06 21:03:56.897001 ip-10-0-128-47 kubenswrapper[2554]: E0706 21:03:56.896979    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4\": container with ID starting with 82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4 not found: ID does not exist" containerID="82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"
Jul 06 21:03:56.897052 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.897010    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4"} err="failed to get container status \"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4\": rpc error: code = NotFound desc = could not find container \"82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4\": container with ID starting with 82f0760f7970adbb84e02123cd38bad33f176f12772f00fb0ad2860c711405f4 not found: ID does not exist"
Jul 06 21:03:56.910409 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.910380    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:03:56.915538 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.915509    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-544d30/sub-test-model-544d30-kserve-86478ffcd7-qslmw"]
Jul 06 21:03:56.928886 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.928863    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zstrr\" (UniqueName: \"kubernetes.io/projected/1794830b-6c81-42aa-8227-2042d9425434-kube-api-access-zstrr\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:56.928886 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.928885    2554 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-model-cache\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:56.929038 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.928896    2554 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1794830b-6c81-42aa-8227-2042d9425434-dshm\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:56.929038 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:56.928905    2554 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1794830b-6c81-42aa-8227-2042d9425434-tls-certs\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:03:57.281877 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:03:57.281847    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1794830b-6c81-42aa-8227-2042d9425434" path="/var/lib/kubelet/pods/1794830b-6c81-42aa-8227-2042d9425434/volumes"
Jul 06 21:05:24.698316 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:24.698230    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 21:05:24.698785 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:24.698453    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" podUID="c12f1a37-e6db-4646-a52d-7b955232c7e7" containerName="maas-api" containerID="cri-o://a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620" gracePeriod=30
Jul 06 21:05:24.739502 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:24.739468    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"]
Jul 06 21:05:24.744814 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:24.744790    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-a-544d30-29722860-69rtf"]
Jul 06 21:05:24.948591 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:24.948515    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 21:05:25.100648 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.100601    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca\") pod \"c12f1a37-e6db-4646-a52d-7b955232c7e7\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") "
Jul 06 21:05:25.100859 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.100688    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls\") pod \"c12f1a37-e6db-4646-a52d-7b955232c7e7\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") "
Jul 06 21:05:25.100859 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.100794    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69cmr\" (UniqueName: \"kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr\") pod \"c12f1a37-e6db-4646-a52d-7b955232c7e7\" (UID: \"c12f1a37-e6db-4646-a52d-7b955232c7e7\") "
Jul 06 21:05:25.101033 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.101002    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c12f1a37-e6db-4646-a52d-7b955232c7e7" (UID: "c12f1a37-e6db-4646-a52d-7b955232c7e7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 21:05:25.102830 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.102800    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c12f1a37-e6db-4646-a52d-7b955232c7e7" (UID: "c12f1a37-e6db-4646-a52d-7b955232c7e7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:05:25.102937 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.102855    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr" (OuterVolumeSpecName: "kube-api-access-69cmr") pod "c12f1a37-e6db-4646-a52d-7b955232c7e7" (UID: "c12f1a37-e6db-4646-a52d-7b955232c7e7"). InnerVolumeSpecName "kube-api-access-69cmr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:05:25.201608 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.201486    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69cmr\" (UniqueName: \"kubernetes.io/projected/c12f1a37-e6db-4646-a52d-7b955232c7e7-kube-api-access-69cmr\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:25.201608 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.201556    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c12f1a37-e6db-4646-a52d-7b955232c7e7-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:25.201608 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.201568    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c12f1a37-e6db-4646-a52d-7b955232c7e7-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:25.281889 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.281860    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2cd8754-793c-4f76-8436-7cdd4d7766ce" path="/var/lib/kubelet/pods/c2cd8754-793c-4f76-8436-7cdd4d7766ce/volumes"
Jul 06 21:05:25.293940 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.293911    2554 generic.go:356] "Generic (PLEG): container finished" podID="c12f1a37-e6db-4646-a52d-7b955232c7e7" containerID="a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620" exitCode=0
Jul 06 21:05:25.294058 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.293949    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"
Jul 06 21:05:25.294058 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.293987    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" event={"ID":"c12f1a37-e6db-4646-a52d-7b955232c7e7","Type":"ContainerDied","Data":"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"}
Jul 06 21:05:25.294058 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.294029    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6" event={"ID":"c12f1a37-e6db-4646-a52d-7b955232c7e7","Type":"ContainerDied","Data":"54503c28ded7596f403b11062322f5019b792382da7c28240067a152080d2681"}
Jul 06 21:05:25.294058 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.294051    2554 scope.go:117] "RemoveContainer" containerID="a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"
Jul 06 21:05:25.303052 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.303033    2554 scope.go:117] "RemoveContainer" containerID="a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"
Jul 06 21:05:25.303291 ip-10-0-128-47 kubenswrapper[2554]: E0706 21:05:25.303272    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620\": container with ID starting with a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620 not found: ID does not exist" containerID="a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"
Jul 06 21:05:25.303339 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.303299    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620"} err="failed to get container status \"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620\": rpc error: code = NotFound desc = could not find container \"a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620\": container with ID starting with a7b78ebe5bd07d55755c9eb96d3f2184acaeae523b90c156bccb4363f3490620 not found: ID does not exist"
Jul 06 21:05:25.312859 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.312833    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 21:05:25.316700 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:25.316679    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-544d30-bb6fd4674-d67g6"]
Jul 06 21:05:27.281431 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:27.281398    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c12f1a37-e6db-4646-a52d-7b955232c7e7" path="/var/lib/kubelet/pods/c12f1a37-e6db-4646-a52d-7b955232c7e7/volumes"
Jul 06 21:05:37.395882 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.395848    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 21:05:37.396390 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.396052    2554 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" podUID="273abb3b-e78b-4148-8460-502a394cfd5a" containerName="maas-api" containerID="cri-o://dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27" gracePeriod=30
Jul 06 21:05:37.431062 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.431033    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"]
Jul 06 21:05:37.433518 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.433488    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-cd7c86-29722860-pbckw"]
Jul 06 21:05:37.644220 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.644190    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 21:05:37.816165 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.816135    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca\") pod \"273abb3b-e78b-4148-8460-502a394cfd5a\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") "
Jul 06 21:05:37.816381 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.816179    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls\") pod \"273abb3b-e78b-4148-8460-502a394cfd5a\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") "
Jul 06 21:05:37.816381 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.816235    2554 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vh9t8\" (UniqueName: \"kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8\") pod \"273abb3b-e78b-4148-8460-502a394cfd5a\" (UID: \"273abb3b-e78b-4148-8460-502a394cfd5a\") "
Jul 06 21:05:37.816612 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.816589    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "273abb3b-e78b-4148-8460-502a394cfd5a" (UID: "273abb3b-e78b-4148-8460-502a394cfd5a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 06 21:05:37.818317 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.818294    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8" (OuterVolumeSpecName: "kube-api-access-vh9t8") pod "273abb3b-e78b-4148-8460-502a394cfd5a" (UID: "273abb3b-e78b-4148-8460-502a394cfd5a"). InnerVolumeSpecName "kube-api-access-vh9t8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 06 21:05:37.818407 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.818388    2554 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "273abb3b-e78b-4148-8460-502a394cfd5a" (UID: "273abb3b-e78b-4148-8460-502a394cfd5a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 06 21:05:37.917448 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.917408    2554 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vh9t8\" (UniqueName: \"kubernetes.io/projected/273abb3b-e78b-4148-8460-502a394cfd5a-kube-api-access-vh9t8\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:37.917448 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.917437    2554 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/273abb3b-e78b-4148-8460-502a394cfd5a-openshift-service-ca\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:37.917448 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:37.917447    2554 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/273abb3b-e78b-4148-8460-502a394cfd5a-maas-api-tls\") on node \"ip-10-0-128-47.ec2.internal\" DevicePath \"\""
Jul 06 21:05:38.357956 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.357919    2554 generic.go:356] "Generic (PLEG): container finished" podID="273abb3b-e78b-4148-8460-502a394cfd5a" containerID="dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27" exitCode=0
Jul 06 21:05:38.358127 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.357969    2554 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"
Jul 06 21:05:38.358127 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.357978    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" event={"ID":"273abb3b-e78b-4148-8460-502a394cfd5a","Type":"ContainerDied","Data":"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"}
Jul 06 21:05:38.358127 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.358032    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l" event={"ID":"273abb3b-e78b-4148-8460-502a394cfd5a","Type":"ContainerDied","Data":"98c4511959641631b6ea00cb919c10b706c3ce40ab4364f5f09ad0759abf969f"}
Jul 06 21:05:38.358127 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.358064    2554 scope.go:117] "RemoveContainer" containerID="dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"
Jul 06 21:05:38.367281 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.367262    2554 scope.go:117] "RemoveContainer" containerID="dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"
Jul 06 21:05:38.367582 ip-10-0-128-47 kubenswrapper[2554]: E0706 21:05:38.367552    2554 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27\": container with ID starting with dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27 not found: ID does not exist" containerID="dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"
Jul 06 21:05:38.367645 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.367592    2554 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27"} err="failed to get container status \"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27\": rpc error: code = NotFound desc = could not find container \"dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27\": container with ID starting with dbb79f1738fd55e6093f431629b2098d2cee75ce73f16b4765a1af4ec64d3e27 not found: ID does not exist"
Jul 06 21:05:38.380757 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.380731    2554 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 21:05:38.388758 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:38.388730    2554 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-cd7c86-5d566fcdc6-g9w5l"]
Jul 06 21:05:39.283688 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:39.283646    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="273abb3b-e78b-4148-8460-502a394cfd5a" path="/var/lib/kubelet/pods/273abb3b-e78b-4148-8460-502a394cfd5a/volumes"
Jul 06 21:05:39.284205 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:39.284048    2554 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b9787b5-14bd-4142-8998-3153117a7c4e" path="/var/lib/kubelet/pods/5b9787b5-14bd-4142-8998-3153117a7c4e/volumes"
Jul 06 21:05:49.773031 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:49.773002    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7ff5fcbbd4-lct6m_985f9f65-be68-4028-9352-b326d5f2ebe7/maas-api/0.log"
Jul 06 21:05:49.895875 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:49.895848    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722845-zlc7l_5812717c-71a0-41a7-8d35-a0fa2b712aaa/cleanup/0.log"
Jul 06 21:05:50.015145 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:50.015109    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722860-9w262_0388722a-1b7c-4bd0-8afe-6a5826ab4c55/cleanup/0.log"
Jul 06 21:05:50.138400 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:50.138324    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-846c44d545-sfr4d_c1e638d5-bce8-43e7-b1fe-1879e4f02fe4/manager/0.log"
Jul 06 21:05:50.818847 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:50.818820    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656f947554-9tkc8_eb185922-ee38-4b6b-8b0e-a66c89bf882e/copy-manifests/0.log"
Jul 06 21:05:50.826443 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:50.826418    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656f947554-9tkc8_eb185922-ee38-4b6b-8b0e-a66c89bf882e/manager/0.log"
Jul 06 21:05:50.943079 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:50.943047    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-47zw5_657366a3-b1da-455f-87d0-db9c2ebd279a/postgres/0.log"
Jul 06 21:05:52.547683 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:52.547655    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7gcjk_fddc4ecc-2c28-46b6-bee6-43dfbded20c4/manager/0.log"
Jul 06 21:05:52.665226 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:52.665199    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8whhz_20b63b5c-fbc4-405d-9ade-fcf1be8a4cda/kuadrant-console-plugin/0.log"
Jul 06 21:05:52.784241 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:52.784214    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5rvq5_eada3974-58f8-45a6-9e42-0f592dc00f50/registry-server/0.log"
Jul 06 21:05:52.915848 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:52.915773    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gxw2w_2efbde4a-6867-4c5e-be6a-782ad29a5a61/manager/0.log"
Jul 06 21:05:54.236303 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:54.236276    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7rdmj_da3d9586-cf9a-4202-844d-ae8bd56a5ae3/payload-pre-processing/0.log"
Jul 06 21:05:54.365944 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:54.365916    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bksg8_950c1428-0dca-4f1d-8ab6-539a0f22742e/payload-processing/0.log"
Jul 06 21:05:54.874977 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:54.874952    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-gsmq7_8d1c9f29-a3b2-403a-a663-1d8e9ff1cfa6/main/0.log"
Jul 06 21:05:55.113804 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:55.113768    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-h29gh_969dbebc-f597-4d81-8f58-35f240ad9f53/main/0.log"
Jul 06 21:05:55.367973 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:05:55.367943    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-njk76_67a6dc92-9b90-48cf-9458-b264c5426fab/main/0.log"
Jul 06 21:06:02.817418 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:02.817389    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-xr6rj_59ae90f0-b70b-4374-90ac-d84276e8bccd/global-pull-secret-syncer/0.log"
Jul 06 21:06:03.021975 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:03.021939    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-x57bd_effb26aa-de4d-48c5-a38a-67a02809854b/konnectivity-agent/0.log"
Jul 06 21:06:03.052940 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:03.052904    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-47.ec2.internal_61ef1a50927e5ff318f1211f698ffaa1/haproxy/0.log"
Jul 06 21:06:08.496256 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:08.496224    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-7gcjk_fddc4ecc-2c28-46b6-bee6-43dfbded20c4/manager/0.log"
Jul 06 21:06:08.552846 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:08.552819    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-8whhz_20b63b5c-fbc4-405d-9ade-fcf1be8a4cda/kuadrant-console-plugin/0.log"
Jul 06 21:06:08.631378 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:08.631349    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-5rvq5_eada3974-58f8-45a6-9e42-0f592dc00f50/registry-server/0.log"
Jul 06 21:06:08.805919 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:08.805876    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-gxw2w_2efbde4a-6867-4c5e-be6a-782ad29a5a61/manager/0.log"
Jul 06 21:06:11.497038 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:11.497003    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfcbb_3b5cb8f1-e084-4148-95fd-8c4702c4a05f/node-exporter/0.log"
Jul 06 21:06:11.531344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:11.531315    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfcbb_3b5cb8f1-e084-4148-95fd-8c4702c4a05f/kube-rbac-proxy/0.log"
Jul 06 21:06:11.572519 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:11.572493    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mfcbb_3b5cb8f1-e084-4148-95fd-8c4702c4a05f/init-textfile/0.log"
Jul 06 21:06:12.996954 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.996917    2554 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"]
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997232    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c12f1a37-e6db-4646-a52d-7b955232c7e7" containerName="maas-api"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997245    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="c12f1a37-e6db-4646-a52d-7b955232c7e7" containerName="maas-api"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997259    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="273abb3b-e78b-4148-8460-502a394cfd5a" containerName="maas-api"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997265    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="273abb3b-e78b-4148-8460-502a394cfd5a" containerName="maas-api"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997282    2554 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1794830b-6c81-42aa-8227-2042d9425434" containerName="main"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997289    2554 state_mem.go:107] "Deleted CPUSet assignment" podUID="1794830b-6c81-42aa-8227-2042d9425434" containerName="main"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997337    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="273abb3b-e78b-4148-8460-502a394cfd5a" containerName="maas-api"
Jul 06 21:06:12.997344 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997344    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="1794830b-6c81-42aa-8227-2042d9425434" containerName="main"
Jul 06 21:06:12.997650 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:12.997351    2554 memory_manager.go:368] "RemoveStaleState removing state" podUID="c12f1a37-e6db-4646-a52d-7b955232c7e7" containerName="maas-api"
Jul 06 21:06:13.000791 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.000770    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.004223 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.004203    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9hcgk\"/\"openshift-service-ca.crt\""
Jul 06 21:06:13.004321 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.004237    2554 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9hcgk\"/\"kube-root-ca.crt\""
Jul 06 21:06:13.005190 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.005173    2554 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9hcgk\"/\"default-dockercfg-mnbcs\""
Jul 06 21:06:13.010718 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.010697    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"]
Jul 06 21:06:13.087338 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.087304    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-lib-modules\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.087338 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.087344    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-sys\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.087588 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.087365    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcxt4\" (UniqueName: \"kubernetes.io/projected/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-kube-api-access-gcxt4\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.087588 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.087395    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-proc\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.087588 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.087423    2554 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-podres\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.187955 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.187912    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-lib-modules\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.187955 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.187961    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-sys\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.187986    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcxt4\" (UniqueName: \"kubernetes.io/projected/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-kube-api-access-gcxt4\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188072    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-sys\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188097    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-lib-modules\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188090    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-proc\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188125    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-proc\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188228 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188149    2554 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-podres\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.188438 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.188283    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-podres\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.197299 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.197272    2554 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcxt4\" (UniqueName: \"kubernetes.io/projected/c307853e-019e-4b63-b8f8-e0a4dfda3fd0-kube-api-access-gcxt4\") pod \"perf-node-gather-daemonset-qgw7j\" (UID: \"c307853e-019e-4b63-b8f8-e0a4dfda3fd0\") " pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.311185 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.311153    2554 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:13.451098 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.451069    2554 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"]
Jul 06 21:06:13.453299 ip-10-0-128-47 kubenswrapper[2554]: W0706 21:06:13.453264    2554 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc307853e_019e_4b63_b8f8_e0a4dfda3fd0.slice/crio-a774edc7423bd1e0e20e0d061c186932e57e3cccfd8738398802e4e96bb43b60 WatchSource:0}: Error finding container a774edc7423bd1e0e20e0d061c186932e57e3cccfd8738398802e4e96bb43b60: Status 404 returned error can't find the container with id a774edc7423bd1e0e20e0d061c186932e57e3cccfd8738398802e4e96bb43b60
Jul 06 21:06:13.522766 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:13.522728    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j" event={"ID":"c307853e-019e-4b63-b8f8-e0a4dfda3fd0","Type":"ContainerStarted","Data":"a774edc7423bd1e0e20e0d061c186932e57e3cccfd8738398802e4e96bb43b60"}
Jul 06 21:06:14.529803 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:14.529762    2554 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j" event={"ID":"c307853e-019e-4b63-b8f8-e0a4dfda3fd0","Type":"ContainerStarted","Data":"da304fe1ba32e8111cf6119d75945107d3520de6e429683bda8e76819a5f3f21"}
Jul 06 21:06:14.530204 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:14.529847    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:14.554317 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:14.554259    2554 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j" podStartSLOduration=2.554241723 podStartE2EDuration="2.554241723s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-06 21:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-06 21:06:14.551198031 +0000 UTC m=+2575.835248572" watchObservedRunningTime="2026-07-06 21:06:14.554241723 +0000 UTC m=+2575.838292264"
Jul 06 21:06:17.421055 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:17.421024    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cfjph_2722582d-cd19-48d0-8c10-0f26029db59b/dns/0.log"
Jul 06 21:06:17.489742 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:17.489707    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-cfjph_2722582d-cd19-48d0-8c10-0f26029db59b/kube-rbac-proxy/0.log"
Jul 06 21:06:17.776521 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:17.776496    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rm27j_1004a321-ebe7-4838-88bf-6c0b45512844/dns-node-resolver/0.log"
Jul 06 21:06:18.531809 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:18.531778    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mjjfd_f616263c-0395-4994-95de-ec5860eac4a7/node-ca/0.log"
Jul 06 21:06:19.432456 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:19.432425    2554 scope.go:117] "RemoveContainer" containerID="43a27d388c5c8cf0e775afd2d4e7d435c653187bea05473bda221c9fba9b4b71"
Jul 06 21:06:19.443735 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:19.443711    2554 scope.go:117] "RemoveContainer" containerID="26f841785ef8be7a870d96a169eb8cebb5fe4509bb7169b5090e424fdcae451b"
Jul 06 21:06:20.018608 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:20.018558    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7rdmj_da3d9586-cf9a-4202-844d-ae8bd56a5ae3/payload-pre-processing/0.log"
Jul 06 21:06:20.119888 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:20.119854    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bksg8_950c1428-0dca-4f1d-8ab6-539a0f22742e/payload-processing/0.log"
Jul 06 21:06:20.544845 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:20.544817    2554 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-9hcgk/perf-node-gather-daemonset-qgw7j"
Jul 06 21:06:20.906415 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:20.906333    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-dsztb_78151588-e645-407d-ab18-f0e5870972f6/serve-healthcheck-canary/0.log"
Jul 06 21:06:21.871170 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:21.871128    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgdml_f1b98fc0-444e-4221-92d9-0eeed9591a1c/kube-rbac-proxy/0.log"
Jul 06 21:06:21.901841 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:21.901814    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgdml_f1b98fc0-444e-4221-92d9-0eeed9591a1c/exporter/0.log"
Jul 06 21:06:21.949102 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:21.949031    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-pgdml_f1b98fc0-444e-4221-92d9-0eeed9591a1c/extractor/0.log"
Jul 06 21:06:24.249587 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:24.249557    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7rdmj_da3d9586-cf9a-4202-844d-ae8bd56a5ae3/payload-pre-processing/0.log"
Jul 06 21:06:24.293028 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:24.292995    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bksg8_950c1428-0dca-4f1d-8ab6-539a0f22742e/payload-processing/0.log"
Jul 06 21:06:24.983689 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:24.983618    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-7ff5fcbbd4-lct6m_985f9f65-be68-4028-9352-b326d5f2ebe7/maas-api/0.log"
Jul 06 21:06:25.008006 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.007983    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722845-zlc7l_5812717c-71a0-41a7-8d35-a0fa2b712aaa/cleanup/0.log"
Jul 06 21:06:25.037916 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.037889    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29722860-9w262_0388722a-1b7c-4bd0-8afe-6a5826ab4c55/cleanup/0.log"
Jul 06 21:06:25.137882 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.137850    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-846c44d545-sfr4d_c1e638d5-bce8-43e7-b1fe-1879e4f02fe4/manager/0.log"
Jul 06 21:06:25.559675 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.559647    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656f947554-9tkc8_eb185922-ee38-4b6b-8b0e-a66c89bf882e/manager/0.log"
Jul 06 21:06:25.599486 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.599456    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-656f947554-9tkc8_eb185922-ee38-4b6b-8b0e-a66c89bf882e/copy-manifests/0.log"
Jul 06 21:06:25.652510 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.652480    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-47zw5_657366a3-b1da-455f-87d0-db9c2ebd279a/postgres/0.log"
Jul 06 21:06:25.791608 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.791577    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-7rdmj_da3d9586-cf9a-4202-844d-ae8bd56a5ae3/payload-pre-processing/0.log"
Jul 06 21:06:25.835419 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:25.835342    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-bksg8_950c1428-0dca-4f1d-8ab6-539a0f22742e/payload-processing/0.log"
Jul 06 21:06:26.966271 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:26.966242    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6cf865c78f-jm5kg_1d79b6fc-4720-4b90-955c-1beb9e81faf9/manager/0.log"
Jul 06 21:06:27.028405 ip-10-0-128-47 kubenswrapper[2554]: I0706 21:06:27.028375    2554 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tt2gs_42dfa53d-03b2-4c6d-89b8-8e6140e7a301/openshift-lws-operator/0.log"