Jun 14 19:11:12.282065 ip-10-0-135-103 systemd[1]: Starting Kubernetes Kubelet... Jun 14 19:11:12.694146 ip-10-0-135-103 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 14 19:11:12.694146 ip-10-0-135-103 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 14 19:11:12.694146 ip-10-0-135-103 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 14 19:11:12.694146 ip-10-0-135-103 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 14 19:11:12.694146 ip-10-0-135-103 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 14 19:11:12.695088 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.695007 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 14 19:11:12.700245 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700231 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.700245 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700245 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700249 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700267 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700270 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700273 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700276 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700279 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700281 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700284 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700287 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700289 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700292 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700295 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700297 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700300 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700303 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700305 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700308 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700310 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.700305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700313 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700316 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700319 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700322 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700324 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700327 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700329 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700332 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700335 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700337 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700340 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700342 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700346 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700350 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700353 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700356 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700364 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700367 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700370 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.700768 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700373 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700376 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700379 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700381 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700384 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700387 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700389 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700392 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700394 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700397 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700399 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700403 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700406 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700409 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700412 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700416 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700419 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700421 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700424 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.701305 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700427 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700429 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700432 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700434 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700437 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700439 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700442 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700447 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700451 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700453 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700456 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700464 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700467 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700470 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700473 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700476 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700479 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700481 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700484 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.701776 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700486 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700489 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700492 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700494 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700497 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700499 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700502 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700505 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700507 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700510 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700513 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700516 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700518 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700521 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700523 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700526 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700528 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700531 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700533 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700537 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.702230 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700539 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.702748 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.700542 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.702972 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702959 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.702972 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702971 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702975 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702978 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702981 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702984 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702987 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702990 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702993 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702995 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.702998 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703001 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703003 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703006 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703008 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703011 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703014 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703016 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703019 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703021 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703024 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.703032 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703027 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703030 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703033 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703036 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703039 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703042 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703044 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703049 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703052 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703054 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703057 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703060 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703062 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703065 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703068 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703071 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703073 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703076 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703078 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703080 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.703534 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703083 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703085 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703088 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703090 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703092 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703095 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703106 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703110 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703113 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703116 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703118 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703121 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703124 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703126 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703130 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703133 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703136 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703139 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703143 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703146 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.704292 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703149 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703151 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703154 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703157 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703160 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703162 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703165 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703168 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703171 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703174 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703176 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703179 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703182 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703184 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703186 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703189 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703191 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703194 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703196 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.704851 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703199 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703201 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703203 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703206 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703208 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703211 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703213 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703218 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703222 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703225 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703227 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703230 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703232 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703235 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703237 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703240 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703242 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703245 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.703247 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.705316 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704110 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704122 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704128 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704133 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704138 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704142 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704153 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704157 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704161 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704166 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704170 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704174 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704179 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704183 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704187 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704193 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704199 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704204 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704213 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.705791 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704218 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704224 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704229 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704234 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704238 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704243 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704247 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704252 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704256 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704261 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704266 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704270 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704279 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704284 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704288 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704293 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704299 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704303 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704307 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704312 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.706250 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704316 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704320 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704324 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704328 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704337 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704341 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704347 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704355 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704360 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704364 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704369 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704374 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704380 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704395 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704407 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704413 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704418 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704422 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704427 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.706745 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704432 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704437 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704446 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704542 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704549 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704553 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704557 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704563 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704567 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704572 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704576 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704580 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704584 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704588 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704592 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704595 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704599 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704603 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704607 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704611 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.707226 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704614 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704618 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704622 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704627 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704645 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704650 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704655 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704659 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704663 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704667 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704671 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704675 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704679 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704683 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704687 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704691 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704695 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704700 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704704 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704708 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.707722 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.704712 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704802 2567 flags.go:64] FLAG: --address="0.0.0.0" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704811 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704821 2567 flags.go:64] FLAG: --anonymous-auth="true" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704828 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704833 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704836 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704840 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704844 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704850 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704853 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704856 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704859 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704863 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704866 2567 flags.go:64] FLAG: --cgroup-root="" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704869 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704872 2567 flags.go:64] FLAG: --client-ca-file="" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704874 2567 flags.go:64] FLAG: --cloud-provider="external" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704877 2567 flags.go:64] FLAG: --cluster-dns="[]" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704881 2567 flags.go:64] FLAG: --cluster-domain="" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704884 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704888 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704891 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704894 2567 flags.go:64] FLAG: --container-log-max-files="5" Jun 14 19:11:12.708237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704897 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704900 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704903 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704906 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704908 2567 flags.go:64] FLAG: --contention-profiling="false" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704911 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704914 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704917 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704920 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704924 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704927 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704929 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704932 2567 flags.go:64] FLAG: --enable-load-reader="false" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704935 2567 flags.go:64] FLAG: --enable-server="true" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704938 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704942 2567 flags.go:64] FLAG: --event-burst="100" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704945 2567 flags.go:64] FLAG: --event-qps="50" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704949 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704952 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704955 2567 flags.go:64] FLAG: --eviction-hard="" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704959 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704961 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704964 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704967 2567 flags.go:64] FLAG: --eviction-soft="" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704970 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 14 19:11:12.708824 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704973 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704976 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704980 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704983 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704985 2567 flags.go:64] FLAG: --fail-swap-on="true" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704988 2567 flags.go:64] FLAG: --feature-gates="" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704992 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704994 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.704997 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705000 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705003 2567 flags.go:64] FLAG: --healthz-port="10248" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705006 2567 flags.go:64] FLAG: --help="false" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705009 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705012 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705015 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705017 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705021 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705024 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705026 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705029 2567 flags.go:64] FLAG: --image-service-endpoint="" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705032 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705035 2567 flags.go:64] FLAG: --kube-api-burst="100" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705037 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705040 2567 flags.go:64] FLAG: --kube-api-qps="50" Jun 14 19:11:12.709435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705045 2567 flags.go:64] FLAG: --kube-reserved="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705047 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705050 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705053 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705056 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705059 2567 flags.go:64] FLAG: --lock-file="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705061 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705067 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705070 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705076 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705078 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705082 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705084 2567 flags.go:64] FLAG: --logging-format="text" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705087 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705091 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705094 2567 flags.go:64] FLAG: --manifest-url="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705096 2567 flags.go:64] FLAG: --manifest-url-header="" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705100 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705103 2567 flags.go:64] FLAG: --max-open-files="1000000" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705108 2567 flags.go:64] FLAG: --max-pods="110" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705111 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705114 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705117 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705120 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705123 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 14 19:11:12.710032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705125 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705128 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705141 2567 flags.go:64] FLAG: --node-status-max-images="50" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705143 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705147 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705150 2567 flags.go:64] FLAG: --pod-cidr="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705152 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705159 2567 flags.go:64] FLAG: --pod-manifest-path="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705162 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705165 2567 flags.go:64] FLAG: --pods-per-core="0" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705168 2567 flags.go:64] FLAG: --port="10250" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705171 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705174 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0121e387d0a5c92b2" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705177 2567 flags.go:64] FLAG: --qos-reserved="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705180 2567 flags.go:64] FLAG: --read-only-port="10255" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705184 2567 flags.go:64] FLAG: --register-node="true" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705187 2567 flags.go:64] FLAG: --register-with-taints="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705194 2567 flags.go:64] FLAG: --registry-burst="10" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705197 2567 flags.go:64] FLAG: --registry-qps="5" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705200 2567 flags.go:64] FLAG: --reserved-cpus="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705203 2567 flags.go:64] FLAG: --reserved-memory="" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705207 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705210 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705213 2567 flags.go:64] FLAG: --rotate-certificates="false" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705216 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 14 19:11:12.710646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705218 2567 flags.go:64] FLAG: --runonce="false" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705221 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705224 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705227 2567 flags.go:64] FLAG: --seccomp-default="false" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705230 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705233 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705235 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705239 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705242 2567 flags.go:64] FLAG: --storage-driver-password="root" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705244 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705247 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705250 2567 flags.go:64] FLAG: --storage-driver-user="root" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705253 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705256 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705260 2567 flags.go:64] FLAG: --system-cgroups="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705263 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705268 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705271 2567 flags.go:64] FLAG: --tls-cert-file="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705274 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705278 2567 flags.go:64] FLAG: --tls-min-version="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705281 2567 flags.go:64] FLAG: --tls-private-key-file="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705284 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705288 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705291 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705294 2567 flags.go:64] FLAG: --v="2" Jun 14 19:11:12.711377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705297 2567 flags.go:64] FLAG: --version="false" Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705301 2567 flags.go:64] FLAG: --vmodule="" Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705305 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.705308 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705413 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705417 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705421 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705423 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705426 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705429 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705433 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705436 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705438 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705441 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705443 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705446 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705448 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705451 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705453 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705456 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.712009 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705459 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705462 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705465 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705467 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705470 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705473 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705476 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705478 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705481 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705484 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705487 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705489 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705492 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705495 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705497 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705500 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705503 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705505 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705508 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705510 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.712499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705513 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705515 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705518 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705521 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705523 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705526 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705528 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705531 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705533 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705536 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705538 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705541 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705544 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705547 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705549 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705552 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705555 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705557 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705560 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.713007 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705562 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705565 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705569 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705571 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705574 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705577 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705579 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705582 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705584 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705587 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705590 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705593 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705595 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705598 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705601 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705606 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705608 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705611 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705613 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705616 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.713456 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705618 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705622 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705626 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705629 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705648 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705651 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705654 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705657 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705661 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705665 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705668 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705671 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705674 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705676 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705681 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705683 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705686 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705689 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705691 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.713965 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705694 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705697 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705699 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705702 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.705705 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.706263 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.712724 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.712739 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.712757 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.712762 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712819 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712824 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712827 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712830 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.714428 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712833 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712837 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712840 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712844 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712846 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712849 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712852 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712854 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712857 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712859 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712862 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712864 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712867 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712870 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712873 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712875 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712878 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712880 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712883 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712885 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.714806 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712888 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712892 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712897 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712901 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712904 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712908 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712911 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712914 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712917 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712919 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712922 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712924 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712927 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712930 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712932 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712936 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712940 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712943 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.715306 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712946 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712948 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712951 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712954 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712956 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712959 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712962 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712964 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712967 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712970 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712972 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712975 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712977 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712980 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712983 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712986 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712988 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712991 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712993 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712996 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.715825 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.712999 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713002 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713005 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713007 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713010 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713012 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713015 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713018 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713021 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713023 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713026 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713029 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713032 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713034 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713037 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713039 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713042 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713044 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713047 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713050 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.716304 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713052 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713055 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713058 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713060 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713063 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713065 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713068 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713071 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713073 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713076 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713078 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713081 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713084 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713087 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713090 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713092 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.716817 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713095 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.713100 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713209 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713214 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713217 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713219 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713222 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713225 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713228 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713231 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713233 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713236 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713238 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713241 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713244 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 14 19:11:12.717256 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713247 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713249 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713252 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713255 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713257 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713260 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713262 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713265 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713268 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713270 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713273 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713275 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713278 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713281 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713284 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713286 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713289 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713292 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713294 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 14 19:11:12.717628 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713297 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713299 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713302 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713304 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713307 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713309 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713312 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713314 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713317 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713319 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713322 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713324 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713328 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713331 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713334 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713337 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713339 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713342 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713345 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713348 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 14 19:11:12.718174 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713351 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713354 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713356 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713359 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713362 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713365 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713368 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713370 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713373 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713376 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713378 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713381 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713384 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713386 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713388 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713391 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713393 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713396 2567 feature_gate.go:328] unrecognized feature gate: Example Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713398 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713400 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 14 19:11:12.718713 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713403 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713406 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713408 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713412 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713415 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713418 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713421 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713423 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713426 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713428 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713431 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713433 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713436 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713438 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713442 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713445 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713448 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713451 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713454 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 14 19:11:12.719349 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713456 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713459 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713462 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713466 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713468 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713471 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713474 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:12.713476 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.713481 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.713595 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.716654 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.717956 2567 server.go:1019] "Starting client certificate rotation" Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.718062 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 14 19:11:12.720097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.718102 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 14 19:11:12.720655 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.720630 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 14 19:11:12.722460 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.722444 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 14 19:11:12.734486 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.734469 2567 log.go:25] "Validated CRI v1 runtime API" Jun 14 19:11:12.739264 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.738972 2567 log.go:25] "Validated CRI v1 image API" Jun 14 19:11:12.740237 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.740222 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 14 19:11:12.742793 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.742776 2567 fs.go:135] Filesystem UUIDs: map[4585eeed-1e64-496e-b4d2-6753edd8a562:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f4b0028a-9d38-4829-9857-053e1fab9c1f:/dev/nvme0n1p3] Jun 14 19:11:12.742842 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.742794 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 14 19:11:12.747319 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.747294 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 14 19:11:12.749747 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.749622 2567 manager.go:216] Machine: {Timestamp:2026-06-14 19:11:12.748474842 +0000 UTC m=+0.348424613 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099704 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2cc923a9c73f8de4c2247eedebea9f SystemUUID:ec2cc923-a9c7-3f8d-e4c2-247eedebea9f BootID:e3833ca6-0799-4908-bb94-7e47ec04c9c7 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ef:bf:c9:60:e7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ef:bf:c9:60:e7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:3e:50:ea:46:ce:15 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 14 19:11:12.749747 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.749744 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 14 19:11:12.749864 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.749816 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 14 19:11:12.752603 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.752580 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 14 19:11:12.752778 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.752607 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-135-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 14 19:11:12.753288 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.753278 2567 topology_manager.go:138] "Creating topology manager with none policy" Jun 14 19:11:12.753321 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.753290 2567 container_manager_linux.go:309] "Creating device plugin manager" Jun 14 19:11:12.753321 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.753298 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 14 19:11:12.753951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.753942 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 14 19:11:12.755218 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.755208 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 14 19:11:12.757981 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.757969 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 14 19:11:12.758100 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758091 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 14 19:11:12.758134 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758118 2567 kubelet.go:486] "Attempting to sync node with API server" Jun 14 19:11:12.758134 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758128 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 14 19:11:12.758190 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758138 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 14 19:11:12.758190 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758147 2567 kubelet.go:398] "Adding apiserver pod source" Jun 14 19:11:12.758190 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758155 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 14 19:11:12.758541 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758528 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 14 19:11:12.758581 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.758553 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 14 19:11:12.760448 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.760433 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 14 19:11:12.761760 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.761745 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 14 19:11:12.761837 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.761783 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 14 19:11:12.761983 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.761971 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 14 19:11:12.762048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.761994 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 14 19:11:12.762048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762003 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 14 19:11:12.762048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762013 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 14 19:11:12.762048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762025 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 14 19:11:12.762048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762033 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 14 19:11:12.762507 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762496 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 14 19:11:12.762554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762509 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 14 19:11:12.762554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762520 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 14 19:11:12.762554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762528 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 14 19:11:12.762554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762542 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 14 19:11:12.762554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.762554 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 14 19:11:12.763325 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.763314 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 14 19:11:12.763373 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.763328 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 14 19:11:12.763867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.763847 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txmjz" Jun 14 19:11:12.766435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.766419 2567 server.go:1268] "Started kubelet" Jun 14 19:11:12.766696 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.766678 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-135-103.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 14 19:11:12.766776 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.766565 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 14 19:11:12.766957 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.766936 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 14 19:11:12.767311 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.767284 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 14 19:11:12.767386 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.767288 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 14 19:11:12.767437 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.767273 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-135-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 14 19:11:12.767404 ip-10-0-135-103 systemd[1]: Started Kubernetes Kubelet. Jun 14 19:11:12.768762 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.768738 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 14 19:11:12.768906 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.768891 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-txmjz" Jun 14 19:11:12.770392 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.770369 2567 server.go:310] "Adding debug handlers to kubelet server" Jun 14 19:11:12.773848 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.773830 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 14 19:11:12.774868 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.774854 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 14 19:11:12.774868 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.774862 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 14 19:11:12.774967 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.774946 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:12.775626 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775490 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 14 19:11:12.775626 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775500 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 14 19:11:12.775626 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775509 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 14 19:11:12.775626 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775589 2567 reconstruct.go:97] "Volume reconstruction finished" Jun 14 19:11:12.775626 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775598 2567 reconciler.go:29] "Reconciler: start to sync state" Jun 14 19:11:12.775891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775726 2567 factory.go:55] Registering systemd factory Jun 14 19:11:12.775891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775741 2567 factory.go:223] Registration of the systemd container factory successfully Jun 14 19:11:12.775993 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775974 2567 factory.go:153] Registering CRI-O factory Jun 14 19:11:12.775993 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.775986 2567 factory.go:223] Registration of the crio container factory successfully Jun 14 19:11:12.776081 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.776031 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 14 19:11:12.776081 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.776063 2567 factory.go:103] Registering Raw factory Jun 14 19:11:12.776200 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.776086 2567 manager.go:1196] Started watching for new ooms in manager Jun 14 19:11:12.776563 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.776547 2567 manager.go:349] Starting recovery of all containers Jun 14 19:11:12.777847 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.777681 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:12.786970 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.786940 2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-135-103.ec2.internal\" not found" node="ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.790096 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.790070 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 14 19:11:12.791221 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.791203 2567 manager.go:354] Recovery completed Jun 14 19:11:12.795497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.795482 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.797226 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797211 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.797275 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797237 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.797275 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797247 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.797720 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797710 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.797787 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797711 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 14 19:11:12.797787 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797733 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 14 19:11:12.797787 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.797748 2567 state_mem.go:36] "Initialized new in-memory state store" Jun 14 19:11:12.798305 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798291 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.798350 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798315 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.798350 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798325 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.798870 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798859 2567 policy_none.go:49] "None policy: Start" Jun 14 19:11:12.798911 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798875 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 14 19:11:12.798911 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.798885 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 14 19:11:12.799816 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.799804 2567 policy_none.go:47] "Start" Jun 14 19:11:12.831206 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831188 2567 manager.go:337] "Starting Device Plugin manager" Jun 14 19:11:12.831278 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.831221 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 14 19:11:12.831278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831233 2567 server.go:87] "Starting device plugin registration server" Jun 14 19:11:12.831462 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831451 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 14 19:11:12.831515 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831468 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 14 19:11:12.831648 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831617 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 14 19:11:12.831876 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831734 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 14 19:11:12.831876 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.831750 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 14 19:11:12.832155 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.832137 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 14 19:11:12.832221 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.832188 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:12.846659 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.846644 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 14 19:11:12.849522 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.846683 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 14 19:11:12.849522 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.846707 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jun 14 19:11:12.849522 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.846743 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 14 19:11:12.849522 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.849324 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:12.931813 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.931795 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.932679 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.932663 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.932759 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.932689 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.932759 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.932700 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.932759 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.932720 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.942611 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.942595 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.942685 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.942613 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-135-103.ec2.internal\": node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:12.946995 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.946950 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal"] Jun 14 19:11:12.947089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.947008 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.948257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.948243 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.948337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.948274 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.948337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.948288 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.949422 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.949406 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.949514 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.949500 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.949570 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.949523 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.950086 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950057 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.950086 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950082 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.950086 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950058 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.950086 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950091 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.950320 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950115 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.950320 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.950133 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.951182 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.951166 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.951263 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.951196 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 14 19:11:12.951778 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.951761 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientMemory" Jun 14 19:11:12.951869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.951786 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasNoDiskPressure" Jun 14 19:11:12.951869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:12.951800 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeHasSufficientPID" Jun 14 19:11:12.958018 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.958000 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:12.966153 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.966135 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-103.ec2.internal\" not found" node="ip-10-0-135-103.ec2.internal" Jun 14 19:11:12.970340 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:12.970326 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-135-103.ec2.internal\" not found" node="ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.058993 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.058975 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.076620 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.076604 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.076704 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.076625 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d515948fb06c638da248f4a0591134e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-103.ec2.internal\" (UID: \"d515948fb06c638da248f4a0591134e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.076704 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.076656 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.159198 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.159168 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.177487 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177470 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d515948fb06c638da248f4a0591134e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-103.ec2.internal\" (UID: \"d515948fb06c638da248f4a0591134e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.177554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177492 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.177554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177520 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.177554 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177546 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.177679 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177565 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1353bbcf2cbcc8e5cd34fff5ee4d633f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal\" (UID: \"1353bbcf2cbcc8e5cd34fff5ee4d633f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.177679 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.177598 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d515948fb06c638da248f4a0591134e9-config\") pod \"kube-apiserver-proxy-ip-10-0-135-103.ec2.internal\" (UID: \"d515948fb06c638da248f4a0591134e9\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.259789 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.259728 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.267844 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.267828 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.273840 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.273820 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:13.360395 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.360371 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.460909 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.460883 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.561417 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.561354 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.661907 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.661883 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.717330 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.717308 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 14 19:11:13.717900 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.717453 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 14 19:11:13.717900 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.717479 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 14 19:11:13.759650 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:13.759608 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1353bbcf2cbcc8e5cd34fff5ee4d633f.slice/crio-d86c5b3d5c3a81482b0a7bd2aa72ce489a6cb07bdd451087d3168e70631e6e97 WatchSource:0}: Error finding container d86c5b3d5c3a81482b0a7bd2aa72ce489a6cb07bdd451087d3168e70631e6e97: Status 404 returned error can't find the container with id d86c5b3d5c3a81482b0a7bd2aa72ce489a6cb07bdd451087d3168e70631e6e97 Jun 14 19:11:13.759867 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:13.759849 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd515948fb06c638da248f4a0591134e9.slice/crio-7e37b516930a727c2f4c3554ce5195130ab417e0d0f8226c3c167ae9d50d0b45 WatchSource:0}: Error finding container 7e37b516930a727c2f4c3554ce5195130ab417e0d0f8226c3c167ae9d50d0b45: Status 404 returned error can't find the container with id 7e37b516930a727c2f4c3554ce5195130ab417e0d0f8226c3c167ae9d50d0b45 Jun 14 19:11:13.761962 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.761941 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.764056 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.764044 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 14 19:11:13.770549 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.770523 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-13 19:06:12 +0000 UTC" deadline="2028-02-21 04:51:25.043630842 +0000 UTC" Jun 14 19:11:13.770549 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.770547 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14793h40m11.27308646s" Jun 14 19:11:13.851820 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.851736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" event={"ID":"d515948fb06c638da248f4a0591134e9","Type":"ContainerStarted","Data":"7e37b516930a727c2f4c3554ce5195130ab417e0d0f8226c3c167ae9d50d0b45"} Jun 14 19:11:13.852880 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.852859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" event={"ID":"1353bbcf2cbcc8e5cd34fff5ee4d633f","Type":"ContainerStarted","Data":"d86c5b3d5c3a81482b0a7bd2aa72ce489a6cb07bdd451087d3168e70631e6e97"} Jun 14 19:11:13.863023 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.863000 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:13.879157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:13.879138 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:13.963337 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:13.963315 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:14.063847 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.063825 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-135-103.ec2.internal\" not found" Jun 14 19:11:14.145086 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.144999 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:14.175675 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.175653 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" Jun 14 19:11:14.185357 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.185339 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 14 19:11:14.186413 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.186396 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" Jun 14 19:11:14.195509 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.195485 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 14 19:11:14.759183 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.759146 2567 apiserver.go:52] "Watching apiserver" Jun 14 19:11:14.767586 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.767561 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 14 19:11:14.767994 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.767972 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-4fq4k","openshift-ovn-kubernetes/ovnkube-node-lgwdf","kube-system/global-pull-secret-syncer-9zd6p","kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal","openshift-cluster-node-tuning-operator/tuned-m8cbn","openshift-dns/node-resolver-695qr","openshift-image-registry/node-ca-h6m4m","openshift-multus/multus-gmcf8","openshift-multus/network-metrics-daemon-nr9nh","openshift-network-diagnostics/network-check-target-fhjvt","kube-system/konnectivity-agent-qgnw7","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal","openshift-multus/multus-additional-cni-plugins-pwt78"] Jun 14 19:11:14.769430 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.769406 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.770570 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.770549 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.771727 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.771708 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.772126 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.772104 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.772226 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.772106 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4mtx7\"" Jun 14 19:11:14.772516 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.772486 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 14 19:11:14.772516 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.772517 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.772983 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.772935 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.773286 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.773265 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 14 19:11:14.773367 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.773265 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.774564 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.774259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.774757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.774733 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-28ddp\"" Jun 14 19:11:14.774757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.774750 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 14 19:11:14.774921 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.774832 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 14 19:11:14.774977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.774961 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.775169 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775149 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 14 19:11:14.775252 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775206 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rqnb8\"" Jun 14 19:11:14.775252 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775232 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-v28xn\"" Jun 14 19:11:14.775252 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775244 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 14 19:11:14.775397 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775363 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.775397 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775363 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 14 19:11:14.775484 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775464 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 14 19:11:14.775618 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775599 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.775727 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.775651 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 14 19:11:14.775727 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.775681 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:14.776268 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.776008 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.776453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.776437 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-6knxn\"" Jun 14 19:11:14.777219 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.777190 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:14.777320 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.777265 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:14.778529 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.778510 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.778731 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.778712 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.779599 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.779264 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.781020 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.780998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.782243 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.782223 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.782333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.782269 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 14 19:11:14.782437 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.782344 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-fxdpp\"" Jun 14 19:11:14.782492 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.782434 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.782492 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.782457 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 14 19:11:14.783491 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.783472 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-t6kv2\"" Jun 14 19:11:14.783581 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.783534 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.783660 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.783591 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:14.783774 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.783756 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.783821 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.783792 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.784903 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.784882 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5655dd8a-5500-41b3-add1-0dc28a18292f-host\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.785009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.784917 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-kubelet\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.784943 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-log-socket\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.784967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.784992 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-script-lib\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785016 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-sys\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785039 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-systemd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785061 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-var-lib-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785097 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-etc-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785136 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-bin\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785187 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c89bece3-08c0-4010-88ee-3c98ee534fea-agent-certs\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785218 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovn-node-metrics-cert\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785245 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sh24d\" (UniqueName: \"kubernetes.io/projected/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-kube-api-access-sh24d\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-kubernetes\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785299 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sw77h\" (UniqueName: \"kubernetes.io/projected/3352e235-d393-416f-bcdc-244d965fac9c-kube-api-access-sw77h\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785344 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-systemd-units\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785371 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785397 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c893177-7e5b-47c9-90ee-fdacb90c7720-host-slash\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785418 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785418 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-systemd\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-lib-modules\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785473 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-host\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-tmp\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785521 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsr4n\" (UniqueName: \"kubernetes.io/projected/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-kube-api-access-nsr4n\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785545 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-ovn\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785569 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-env-overrides\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785615 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c893177-7e5b-47c9-90ee-fdacb90c7720-iptables-alerter-script\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785663 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cx7rr\" (UniqueName: \"kubernetes.io/projected/8c893177-7e5b-47c9-90ee-fdacb90c7720-kube-api-access-cx7rr\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785687 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-conf\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785708 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-run\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-var-lib-kubelet\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785810 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-etc-tuned\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785851 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-slash\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785906 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785935 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-netd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.785985 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62hhn\" (UniqueName: \"kubernetes.io/projected/5655dd8a-5500-41b3-add1-0dc28a18292f-kube-api-access-62hhn\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.785999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786007 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-netns\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786029 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-modprobe-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786049 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786073 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786102 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5655dd8a-5500-41b3-add1-0dc28a18292f-serviceca\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-node-log\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786152 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-config\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786176 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysconfig\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.786718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786222 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c89bece3-08c0-4010-88ee-3c98ee534fea-konnectivity-ca\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.787174 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.786743 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.787748 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.787730 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 14 19:11:14.787827 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.787780 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 14 19:11:14.787916 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.787898 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-pf2ld\"" Jun 14 19:11:14.788009 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.787960 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 14 19:11:14.788531 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.788512 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 14 19:11:14.789114 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.789096 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 14 19:11:14.789202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.789145 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 14 19:11:14.789202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.789170 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-wk2xz\"" Jun 14 19:11:14.810063 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.810039 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjn2s" Jun 14 19:11:14.819424 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.819403 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kjn2s" Jun 14 19:11:14.874000 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.873978 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:14.876253 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.876234 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 14 19:11:14.886666 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886628 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5655dd8a-5500-41b3-add1-0dc28a18292f-host\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.886757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886679 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-kubelet\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.886757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-script-lib\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.886757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886739 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-kubelet\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.886867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886762 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-kubelet-config\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.886867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886785 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5655dd8a-5500-41b3-add1-0dc28a18292f-host\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.886867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886789 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-os-release\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.886867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886817 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-systemd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.886867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-var-lib-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886879 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-bin\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886890 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-systemd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886902 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-cni-binary-copy\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886954 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-bin\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886933 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-var-lib-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.886984 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c89bece3-08c0-4010-88ee-3c98ee534fea-agent-certs\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.887055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovn-node-metrics-cert\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887071 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sh24d\" (UniqueName: \"kubernetes.io/projected/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-kube-api-access-sh24d\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887087 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vngms\" (UniqueName: \"kubernetes.io/projected/feee0d5e-a065-4517-b44a-82c38c520f6b-kube-api-access-vngms\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887143 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-systemd-units\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887312 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-systemd\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887332 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-systemd-units\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-lib-modules\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887363 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887381 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-tmp\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsr4n\" (UniqueName: \"kubernetes.io/projected/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-kube-api-access-nsr4n\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887488 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-systemd\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.887497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-hostroot\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887538 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-device-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887619 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-lib-modules\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887703 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-624b7\" (UniqueName: \"kubernetes.io/projected/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-kube-api-access-624b7\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887735 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887794 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-env-overrides\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887837 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-script-lib\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c893177-7e5b-47c9-90ee-fdacb90c7720-iptables-alerter-script\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.887965 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-tmp-dir\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888006 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-k8s-cni-cncf-io\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-dbus\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-netns\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888147 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-env-overrides\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888183 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888224 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-system-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-os-release\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-conf-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888304 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-netns\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888453 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888421 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888468 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-node-log\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888501 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysconfig\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888559 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysconfig\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888559 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-node-log\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888598 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-netns\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888596 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8c893177-7e5b-47c9-90ee-fdacb90c7720-iptables-alerter-script\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888627 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-registration-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-log-socket\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888720 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-sys\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888750 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-log-socket\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888769 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-system-cni-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888795 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888796 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888821 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-etc-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888844 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-multus-daemon-config\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.888888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888853 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-sys\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-multus-certs\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888892 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-etc-openvswitch\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888893 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-kubernetes\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888930 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sw77h\" (UniqueName: \"kubernetes.io/projected/3352e235-d393-416f-bcdc-244d965fac9c-kube-api-access-sw77h\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888958 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-kubernetes\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888960 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-kubelet\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.888988 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-etc-kubernetes\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889119 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889150 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889172 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-hosts-file\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c893177-7e5b-47c9-90ee-fdacb90c7720-host-slash\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-host\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889269 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-ovn\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cx7rr\" (UniqueName: \"kubernetes.io/projected/8c893177-7e5b-47c9-90ee-fdacb90c7720-kube-api-access-cx7rr\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889290 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-run-ovn-kubernetes\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.889694 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889349 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-host\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889396 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-run-ovn\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889430 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-conf\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889441 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8c893177-7e5b-47c9-90ee-fdacb90c7720-host-slash\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889475 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-run\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-var-lib-kubelet\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889531 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-etc-tuned\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889545 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-run\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889553 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-slash\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889565 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-sysctl-conf\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889576 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-netd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889592 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-var-lib-kubelet\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889610 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-cnibin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889647 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-slash\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889651 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-multus\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889683 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-sys-fs\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889685 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-host-cni-netd\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.890432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889708 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kzcr\" (UniqueName: \"kubernetes.io/projected/2ee89d03-f544-4b2f-8626-00b40bae5919-kube-api-access-6kzcr\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889734 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62hhn\" (UniqueName: \"kubernetes.io/projected/5655dd8a-5500-41b3-add1-0dc28a18292f-kube-api-access-62hhn\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889757 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-modprobe-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889778 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-socket-dir-parent\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-socket-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889836 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889862 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5655dd8a-5500-41b3-add1-0dc28a18292f-serviceca\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889910 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-config\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889919 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/3352e235-d393-416f-bcdc-244d965fac9c-etc-modprobe-d\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889944 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.889978 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-bin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.890020 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/91037120-5701-470f-b5da-eef943987ef8-kube-api-access-l64wj\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.890552 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.890666 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:15.39061259 +0000 UTC m=+2.990562371 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891000 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-tmp\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.891202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891089 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-cnibin\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.891960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.891960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891166 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c89bece3-08c0-4010-88ee-3c98ee534fea-konnectivity-ca\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.891960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891534 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovnkube-config\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.891960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891724 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c89bece3-08c0-4010-88ee-3c98ee534fea-konnectivity-ca\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.891960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891876 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-ovn-node-metrics-cert\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.892195 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.891533 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5655dd8a-5500-41b3-add1-0dc28a18292f-serviceca\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.892263 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.892221 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c89bece3-08c0-4010-88ee-3c98ee534fea-agent-certs\") pod \"konnectivity-agent-qgnw7\" (UID: \"c89bece3-08c0-4010-88ee-3c98ee534fea\") " pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:14.892504 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.892484 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/3352e235-d393-416f-bcdc-244d965fac9c-etc-tuned\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.897560 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.897184 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsr4n\" (UniqueName: \"kubernetes.io/projected/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-kube-api-access-nsr4n\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:14.897560 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.897487 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:14.897560 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.897513 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:14.897560 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.897512 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cx7rr\" (UniqueName: \"kubernetes.io/projected/8c893177-7e5b-47c9-90ee-fdacb90c7720-kube-api-access-cx7rr\") pod \"iptables-alerter-4fq4k\" (UID: \"8c893177-7e5b-47c9-90ee-fdacb90c7720\") " pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:14.897560 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.897527 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:14.897788 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.897518 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sh24d\" (UniqueName: \"kubernetes.io/projected/6d3cbe39-23e9-402d-b16f-b19dac8d1e1b-kube-api-access-sh24d\") pod \"ovnkube-node-lgwdf\" (UID: \"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b\") " pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:14.898299 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.898146 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:15.398126725 +0000 UTC m=+2.998076498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:14.899007 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.898980 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-62hhn\" (UniqueName: \"kubernetes.io/projected/5655dd8a-5500-41b3-add1-0dc28a18292f-kube-api-access-62hhn\") pod \"node-ca-h6m4m\" (UID: \"5655dd8a-5500-41b3-add1-0dc28a18292f\") " pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:14.899820 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.899802 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sw77h\" (UniqueName: \"kubernetes.io/projected/3352e235-d393-416f-bcdc-244d965fac9c-kube-api-access-sw77h\") pod \"tuned-m8cbn\" (UID: \"3352e235-d393-416f-bcdc-244d965fac9c\") " pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:14.992277 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992249 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-k8s-cni-cncf-io\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992277 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992279 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-dbus\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-system-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-os-release\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-conf-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992353 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-netns\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992397 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-k8s-cni-cncf-io\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992403 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-netns\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992428 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-os-release\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992435 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-system-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992453 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-dbus\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-registration-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992438 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-conf-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992501 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-registration-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992526 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-system-cni-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992570 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992615 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-system-cni-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992657 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-multus-daemon-config\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992682 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-multus-certs\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992720 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-run-multus-certs\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992743 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-kubelet\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992768 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-etc-kubernetes\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992817 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-hosts-file\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992847 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-kubelet\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992872 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-cnibin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.992895 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992897 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-multus\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992919 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-sys-fs\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992932 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-etc-kubernetes\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992942 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kzcr\" (UniqueName: \"kubernetes.io/projected/2ee89d03-f544-4b2f-8626-00b40bae5919-kube-api-access-6kzcr\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992979 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.992983 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-socket-dir-parent\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993012 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-socket-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993023 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-hosts-file\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993036 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993068 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-multus\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993085 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-bin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993108 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/91037120-5701-470f-b5da-eef943987ef8-kube-api-access-l64wj\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993117 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-socket-dir-parent\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.992939 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993152 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-cnibin\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993195 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-sys-fs\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.993691 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:14.993200 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:15.493161623 +0000 UTC m=+3.093111381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993215 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-cnibin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993221 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-multus-daemon-config\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993245 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993253 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-cnibin\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993277 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-socket-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993296 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-host-var-lib-cni-bin\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993313 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-kubelet-config\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993332 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-etc-selinux\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-os-release\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993379 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-cni-binary-copy\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993380 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ca26259f-d764-4620-bf6d-be60ee15e22e-kubelet-config\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993400 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993404 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-os-release\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vngms\" (UniqueName: \"kubernetes.io/projected/feee0d5e-a065-4517-b44a-82c38c520f6b-kube-api-access-vngms\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993467 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-hostroot\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994327 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993531 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2ee89d03-f544-4b2f-8626-00b40bae5919-tuning-conf-dir\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993537 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-device-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993568 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-624b7\" (UniqueName: \"kubernetes.io/projected/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-kube-api-access-624b7\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993578 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/feee0d5e-a065-4517-b44a-82c38c520f6b-device-dir\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993596 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993604 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-hostroot\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993621 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-tmp-dir\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993677 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91037120-5701-470f-b5da-eef943987ef8-multus-cni-dir\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993704 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993821 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91037120-5701-470f-b5da-eef943987ef8-cni-binary-copy\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993896 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-tmp-dir\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:14.994891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:14.993961 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2ee89d03-f544-4b2f-8626-00b40bae5919-cni-binary-copy\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:15.001415 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.001395 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vngms\" (UniqueName: \"kubernetes.io/projected/feee0d5e-a065-4517-b44a-82c38c520f6b-kube-api-access-vngms\") pod \"aws-ebs-csi-driver-node-z2nld\" (UID: \"feee0d5e-a065-4517-b44a-82c38c520f6b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:15.001630 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.001604 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-624b7\" (UniqueName: \"kubernetes.io/projected/51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e-kube-api-access-624b7\") pod \"node-resolver-695qr\" (UID: \"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e\") " pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:15.002378 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.002356 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l64wj\" (UniqueName: \"kubernetes.io/projected/91037120-5701-470f-b5da-eef943987ef8-kube-api-access-l64wj\") pod \"multus-gmcf8\" (UID: \"91037120-5701-470f-b5da-eef943987ef8\") " pod="openshift-multus/multus-gmcf8" Jun 14 19:11:15.002727 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.002710 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kzcr\" (UniqueName: \"kubernetes.io/projected/2ee89d03-f544-4b2f-8626-00b40bae5919-kube-api-access-6kzcr\") pod \"multus-additional-cni-plugins-pwt78\" (UID: \"2ee89d03-f544-4b2f-8626-00b40bae5919\") " pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:15.082689 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.082613 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-h6m4m" Jun 14 19:11:15.091551 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.091518 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:15.098393 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.098354 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:15.105007 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.104990 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4fq4k" Jun 14 19:11:15.111546 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.111528 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" Jun 14 19:11:15.119309 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.119291 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmcf8" Jun 14 19:11:15.126766 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.126750 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-695qr" Jun 14 19:11:15.134348 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.134329 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" Jun 14 19:11:15.136091 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.136075 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 14 19:11:15.140198 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.140180 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-pwt78" Jun 14 19:11:15.395690 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.395615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:15.395821 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.395778 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:15.395870 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.395841 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:16.39582222 +0000 UTC m=+3.995771979 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:15.496227 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.496199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:15.496357 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.496272 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:15.496357 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496349 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:15.496475 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496374 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:15.496475 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496388 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:15.496475 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496432 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:16.49641641 +0000 UTC m=+4.096366174 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:15.496475 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496351 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:15.496475 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:15.496470 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:16.496461966 +0000 UTC m=+4.096411725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:15.522432 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.522207 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91037120_5701_470f_b5da_eef943987ef8.slice/crio-2457b2640f4dc79e83cd6a4b44ef1146ae30c548361ba2a5038fbc23261925aa WatchSource:0}: Error finding container 2457b2640f4dc79e83cd6a4b44ef1146ae30c548361ba2a5038fbc23261925aa: Status 404 returned error can't find the container with id 2457b2640f4dc79e83cd6a4b44ef1146ae30c548361ba2a5038fbc23261925aa Jun 14 19:11:15.523252 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.523222 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89bece3_08c0_4010_88ee_3c98ee534fea.slice/crio-20068dd31329998927939de16a5519ef534518313f4de31d7ee0b54b6bd062d6 WatchSource:0}: Error finding container 20068dd31329998927939de16a5519ef534518313f4de31d7ee0b54b6bd062d6: Status 404 returned error can't find the container with id 20068dd31329998927939de16a5519ef534518313f4de31d7ee0b54b6bd062d6 Jun 14 19:11:15.524061 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.524021 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeee0d5e_a065_4517_b44a_82c38c520f6b.slice/crio-bc389f91aa0c3f4a49f321633ea01d58f0c5dab6c74fa7ed719b002066a1932e WatchSource:0}: Error finding container bc389f91aa0c3f4a49f321633ea01d58f0c5dab6c74fa7ed719b002066a1932e: Status 404 returned error can't find the container with id bc389f91aa0c3f4a49f321633ea01d58f0c5dab6c74fa7ed719b002066a1932e Jun 14 19:11:15.524730 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.524708 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ee89d03_f544_4b2f_8626_00b40bae5919.slice/crio-a51fa70d9a9dd6ea27064c7f211284b4f91f75c6dc4315f10d86dfd4c1315335 WatchSource:0}: Error finding container a51fa70d9a9dd6ea27064c7f211284b4f91f75c6dc4315f10d86dfd4c1315335: Status 404 returned error can't find the container with id a51fa70d9a9dd6ea27064c7f211284b4f91f75c6dc4315f10d86dfd4c1315335 Jun 14 19:11:15.528228 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.528206 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c893177_7e5b_47c9_90ee_fdacb90c7720.slice/crio-cb825c95d2b39fd512a4ac5083bae2c9c9db75cda476f0d72f05b6729692e965 WatchSource:0}: Error finding container cb825c95d2b39fd512a4ac5083bae2c9c9db75cda476f0d72f05b6729692e965: Status 404 returned error can't find the container with id cb825c95d2b39fd512a4ac5083bae2c9c9db75cda476f0d72f05b6729692e965 Jun 14 19:11:15.529153 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.529098 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3352e235_d393_416f_bcdc_244d965fac9c.slice/crio-7f89635748d3db6d705f1c5170721e18b042cea44a40c6053185a99a1cc08848 WatchSource:0}: Error finding container 7f89635748d3db6d705f1c5170721e18b042cea44a40c6053185a99a1cc08848: Status 404 returned error can't find the container with id 7f89635748d3db6d705f1c5170721e18b042cea44a40c6053185a99a1cc08848 Jun 14 19:11:15.530077 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.530054 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d3cbe39_23e9_402d_b16f_b19dac8d1e1b.slice/crio-e349899c64a2b9bca8e9de067ed91d4a640422d233beb74cbe98ce368a7d2a8c WatchSource:0}: Error finding container e349899c64a2b9bca8e9de067ed91d4a640422d233beb74cbe98ce368a7d2a8c: Status 404 returned error can't find the container with id e349899c64a2b9bca8e9de067ed91d4a640422d233beb74cbe98ce368a7d2a8c Jun 14 19:11:15.530897 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.530864 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51d8c3c1_c3e3_4f68_9af5_a36bfcf3201e.slice/crio-d736ccd82a84519421634c38d299e00ada09a4703c04529a21e71ec48aa26465 WatchSource:0}: Error finding container d736ccd82a84519421634c38d299e00ada09a4703c04529a21e71ec48aa26465: Status 404 returned error can't find the container with id d736ccd82a84519421634c38d299e00ada09a4703c04529a21e71ec48aa26465 Jun 14 19:11:15.532154 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:15.532077 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5655dd8a_5500_41b3_add1_0dc28a18292f.slice/crio-c280bfdb4807ee1754ff29719ec50a63dcf5ee75f20e7e97b982b4674502fa9e WatchSource:0}: Error finding container c280bfdb4807ee1754ff29719ec50a63dcf5ee75f20e7e97b982b4674502fa9e: Status 404 returned error can't find the container with id c280bfdb4807ee1754ff29719ec50a63dcf5ee75f20e7e97b982b4674502fa9e Jun 14 19:11:15.821717 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.821688 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-13 19:06:14 +0000 UTC" deadline="2028-01-19 23:26:48.052212697 +0000 UTC" Jun 14 19:11:15.821717 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.821714 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14020h15m32.230501271s" Jun 14 19:11:15.864882 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.864588 2567 generic.go:356] "Generic (PLEG): container finished" podID="1353bbcf2cbcc8e5cd34fff5ee4d633f" containerID="2b0d33e702df15424652e3cedec60bf7f9cf205128f67a576105909b192cf0c0" exitCode=0 Jun 14 19:11:15.865047 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.865017 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" event={"ID":"1353bbcf2cbcc8e5cd34fff5ee4d633f","Type":"ContainerDied","Data":"2b0d33e702df15424652e3cedec60bf7f9cf205128f67a576105909b192cf0c0"} Jun 14 19:11:15.871332 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.871302 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6m4m" event={"ID":"5655dd8a-5500-41b3-add1-0dc28a18292f","Type":"ContainerStarted","Data":"c280bfdb4807ee1754ff29719ec50a63dcf5ee75f20e7e97b982b4674502fa9e"} Jun 14 19:11:15.874848 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.874821 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-695qr" event={"ID":"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e","Type":"ContainerStarted","Data":"d736ccd82a84519421634c38d299e00ada09a4703c04529a21e71ec48aa26465"} Jun 14 19:11:15.881123 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.880833 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"e349899c64a2b9bca8e9de067ed91d4a640422d233beb74cbe98ce368a7d2a8c"} Jun 14 19:11:15.886590 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.886524 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" event={"ID":"3352e235-d393-416f-bcdc-244d965fac9c","Type":"ContainerStarted","Data":"7f89635748d3db6d705f1c5170721e18b042cea44a40c6053185a99a1cc08848"} Jun 14 19:11:15.888217 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.888143 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4fq4k" event={"ID":"8c893177-7e5b-47c9-90ee-fdacb90c7720","Type":"ContainerStarted","Data":"cb825c95d2b39fd512a4ac5083bae2c9c9db75cda476f0d72f05b6729692e965"} Jun 14 19:11:15.892909 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.892843 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerStarted","Data":"a51fa70d9a9dd6ea27064c7f211284b4f91f75c6dc4315f10d86dfd4c1315335"} Jun 14 19:11:15.894426 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.894365 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" event={"ID":"feee0d5e-a065-4517-b44a-82c38c520f6b","Type":"ContainerStarted","Data":"bc389f91aa0c3f4a49f321633ea01d58f0c5dab6c74fa7ed719b002066a1932e"} Jun 14 19:11:15.898090 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.896157 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qgnw7" event={"ID":"c89bece3-08c0-4010-88ee-3c98ee534fea","Type":"ContainerStarted","Data":"20068dd31329998927939de16a5519ef534518313f4de31d7ee0b54b6bd062d6"} Jun 14 19:11:15.900923 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.899918 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmcf8" event={"ID":"91037120-5701-470f-b5da-eef943987ef8","Type":"ContainerStarted","Data":"2457b2640f4dc79e83cd6a4b44ef1146ae30c548361ba2a5038fbc23261925aa"} Jun 14 19:11:15.904618 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:15.904093 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" event={"ID":"d515948fb06c638da248f4a0591134e9","Type":"ContainerStarted","Data":"357cff6237e86a8a73b2d48531c2ba2740f3174dbd9b21aea2678f218d757778"} Jun 14 19:11:16.406393 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.406309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:16.406605 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.406476 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:16.406605 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.406547 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:18.406527694 +0000 UTC m=+6.006477455 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:16.507917 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.507661 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:16.507917 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.507721 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:16.507917 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.507891 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:16.507917 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.507912 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:16.507917 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.507923 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:16.508258 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.507980 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:18.507963301 +0000 UTC m=+6.107913077 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:16.508877 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.508366 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:16.508877 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.508417 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:18.508402829 +0000 UTC m=+6.108352594 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:16.822921 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.822778 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-13 19:06:14 +0000 UTC" deadline="2028-01-26 22:28:45.309613941 +0000 UTC" Jun 14 19:11:16.822921 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.822840 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14187h17m28.486778458s" Jun 14 19:11:16.849384 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.849358 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:16.849529 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.849496 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:16.849922 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.849900 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:16.850040 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.850005 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:16.850106 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.850070 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:16.850160 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:16.850132 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:16.930748 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.930694 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" event={"ID":"1353bbcf2cbcc8e5cd34fff5ee4d633f","Type":"ContainerStarted","Data":"4129c81a453a9ad43c2dc32b5e68a5b9788fbd3a5a336203466d7a5c16b00124"} Jun 14 19:11:16.946628 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:16.946565 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-135-103.ec2.internal" podStartSLOduration=2.946544873 podStartE2EDuration="2.946544873s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:15.920620618 +0000 UTC m=+3.520570402" watchObservedRunningTime="2026-06-14 19:11:16.946544873 +0000 UTC m=+4.546494657" Jun 14 19:11:18.424010 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.423973 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:18.424440 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.424145 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:18.424440 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.424209 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:22.424191081 +0000 UTC m=+10.024140843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:18.524492 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.524456 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:18.524713 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.524550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:18.524713 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.524707 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:18.524836 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.524770 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:22.524752435 +0000 UTC m=+10.124702198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:18.525179 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.525160 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:18.525252 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.525187 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:18.525252 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.525201 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:18.525252 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.525248 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:22.52523229 +0000 UTC m=+10.125182051 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:18.847813 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.847714 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:18.847813 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.847767 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:18.848095 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:18.847714 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:18.848095 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.847860 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:18.848095 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.847963 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:18.848095 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:18.848057 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:20.847309 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:20.847267 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:20.847748 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:20.847425 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:20.847748 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:20.847274 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:20.847748 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:20.847533 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:20.847748 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:20.847267 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:20.847748 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:20.847619 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:22.461400 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.461360 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:22.462008 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.461502 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:22.462008 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.461575 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:30.461554683 +0000 UTC m=+18.061504514 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:22.562389 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.562344 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:22.562566 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.562441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:22.562566 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562512 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:22.562566 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562536 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:22.562566 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562541 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:22.562566 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562549 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:22.562830 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562601 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:30.562587144 +0000 UTC m=+18.162536907 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:22.562830 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.562621 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:30.562612019 +0000 UTC m=+18.162561778 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:22.848474 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.848324 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:22.848474 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.848457 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:22.849079 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.848919 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:22.849079 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.849017 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:22.849362 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:22.849340 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:22.849456 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:22.849440 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:24.847408 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:24.847370 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:24.847408 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:24.847387 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:24.847997 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:24.847370 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:24.847997 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:24.847492 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:24.847997 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:24.847582 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:24.847997 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:24.847680 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:26.847133 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:26.847100 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:26.847133 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:26.847138 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:26.847667 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:26.847102 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:26.847667 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:26.847227 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:26.847667 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:26.847307 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:26.847667 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:26.847390 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:28.847646 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:28.847610 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:28.848078 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:28.847619 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:28.848078 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:28.847752 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:28.848078 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:28.847845 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:28.848078 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:28.847911 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:28.848078 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:28.847980 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:30.523197 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.523162 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:30.523756 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.523305 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:30.523756 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.523367 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:11:46.523352445 +0000 UTC m=+34.123302204 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:30.623624 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.623590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:30.623808 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.623697 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:30.623808 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623769 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:30.623808 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623794 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:30.623808 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623803 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:30.623956 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623806 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:30.623956 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623854 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:46.623839859 +0000 UTC m=+34.223789623 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:30.623956 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.623869 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:11:46.623862223 +0000 UTC m=+34.223811981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:30.847934 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.847897 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:30.848114 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.847898 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:30.848114 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.848032 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:30.848114 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:30.847914 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:30.848114 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.848094 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:30.848322 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:30.848188 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:32.848149 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:32.848117 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:32.848560 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:32.848210 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:32.848560 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:32.848285 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:32.848560 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:32.848384 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:32.848560 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:32.848456 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:32.848768 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:32.848558 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:34.847189 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.847165 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:34.847913 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.847275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:34.847913 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:34.847287 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:34.847913 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:34.847375 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:34.847913 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.847429 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:34.847913 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:34.847520 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:34.966159 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.965995 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" event={"ID":"3352e235-d393-416f-bcdc-244d965fac9c","Type":"ContainerStarted","Data":"a1ddea923759ce933a23d9f0adf93a69e259c6b1900cca8b7f9a14123afb58d8"} Jun 14 19:11:34.967396 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.967373 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="39bef5d14a750f4e5201afcd635c4fb33a0d5468456c778742aa3293fe6054bb" exitCode=0 Jun 14 19:11:34.967500 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.967456 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"39bef5d14a750f4e5201afcd635c4fb33a0d5468456c778742aa3293fe6054bb"} Jun 14 19:11:34.969066 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.969040 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" event={"ID":"feee0d5e-a065-4517-b44a-82c38c520f6b","Type":"ContainerStarted","Data":"6f0d0ab0cf32b2e1fee5482f1260a9ca8de47e1effddc828d8fc61b7a41f70bd"} Jun 14 19:11:34.970664 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.970623 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-qgnw7" event={"ID":"c89bece3-08c0-4010-88ee-3c98ee534fea","Type":"ContainerStarted","Data":"e40b023cb0208eb6e383a3da34b3105e922abac3db36f91d01f62575d97bdeb1"} Jun 14 19:11:34.972241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.972213 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmcf8" event={"ID":"91037120-5701-470f-b5da-eef943987ef8","Type":"ContainerStarted","Data":"83389cd2cdd169bbc84b4095e18d8d1e3881051006fcb00e85ff32ae44daad2e"} Jun 14 19:11:34.973669 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.973647 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-h6m4m" event={"ID":"5655dd8a-5500-41b3-add1-0dc28a18292f","Type":"ContainerStarted","Data":"6cdfbde3db13b157918f318a7544a5639ed6e39b78407b190d9aa90eeac89bd6"} Jun 14 19:11:34.975321 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.975299 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-695qr" event={"ID":"51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e","Type":"ContainerStarted","Data":"be49c2945afe3cf0791f186dc9d36cea3025d0daf57cd94c3da73fa40181def7"} Jun 14 19:11:34.977447 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.977409 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"2cd50627189ff22a46883ef0a1b75f57ac67c70e50509566e788fb4d988d03c2"} Jun 14 19:11:34.977524 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.977486 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"bd78bfad6d6365b4250d98f8d3fd93e02a52ebc877bfa26d3845d48db4557cc9"} Jun 14 19:11:34.982953 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.982911 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-135-103.ec2.internal" podStartSLOduration=20.98289506 podStartE2EDuration="20.98289506s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:16.947218346 +0000 UTC m=+4.547168128" watchObservedRunningTime="2026-06-14 19:11:34.98289506 +0000 UTC m=+22.582844842" Jun 14 19:11:34.983445 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.983413 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-m8cbn" podStartSLOduration=4.220126005 podStartE2EDuration="22.983402402s" totalImagesPullingTime="18.763276397s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:34.982829994 +0000 UTC m=+22.582779775" watchObservedRunningTime="2026-06-14 19:11:34.983402402 +0000 UTC m=+22.583352184" Jun 14 19:11:34.997434 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:34.997398 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-695qr" podStartSLOduration=7.965254723 podStartE2EDuration="21.997385255s" totalImagesPullingTime="14.032130532s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:34.996772818 +0000 UTC m=+22.596722598" watchObservedRunningTime="2026-06-14 19:11:34.997385255 +0000 UTC m=+22.597335036" Jun 14 19:11:35.047140 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.047101 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-qgnw7" podStartSLOduration=9.007633165 podStartE2EDuration="23.047089521s" totalImagesPullingTime="14.039456356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:35.047003901 +0000 UTC m=+22.646953740" watchObservedRunningTime="2026-06-14 19:11:35.047089521 +0000 UTC m=+22.647039301" Jun 14 19:11:35.047615 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.047590 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-h6m4m" podStartSLOduration=4.288999729 podStartE2EDuration="23.047580743s" totalImagesPullingTime="18.758581014s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:35.032294506 +0000 UTC m=+22.632244306" watchObservedRunningTime="2026-06-14 19:11:35.047580743 +0000 UTC m=+22.647530564" Jun 14 19:11:35.066024 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.065415 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gmcf8" podStartSLOduration=3.28449355 podStartE2EDuration="22.065399142s" totalImagesPullingTime="18.780905592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:35.063909438 +0000 UTC m=+22.663859213" watchObservedRunningTime="2026-06-14 19:11:35.065399142 +0000 UTC m=+22.665348923" Jun 14 19:11:35.782530 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.782504 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 14 19:11:35.844149 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.844035 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-14T19:11:35.782526938Z","UUID":"5a6d6b2d-bdaa-4d9a-a14a-8dbbcb3e955d","Handler":null,"Name":"","Endpoint":""} Jun 14 19:11:35.845577 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.845559 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 14 19:11:35.845705 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.845592 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 14 19:11:35.983108 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.983032 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"470a9f5cd4377969b97e2e6f86fd5c6688b5a0b773e64df7973b3f86d4d4629f"} Jun 14 19:11:35.983584 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.983164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"2158c396eb949c351b71394cd4990482172e0be6f853a77b16d16a56ed917955"} Jun 14 19:11:35.983584 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.983220 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"0a101bcf8c8a78c2d95cab628e93fb70de77e7eccc4db58c3de20389175ffe5b"} Jun 14 19:11:35.983584 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.983269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"e177afba917d420d72495717de02eb634c9cf1ae2ce0cf4846b96dfe8ace3ed6"} Jun 14 19:11:35.984663 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.984615 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4fq4k" event={"ID":"8c893177-7e5b-47c9-90ee-fdacb90c7720","Type":"ContainerStarted","Data":"5617d21f156b489ae0ec44b0b5bf5ab1061eb7fb6978ec56a17b7fb1d30b5246"} Jun 14 19:11:35.986781 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.986755 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" event={"ID":"feee0d5e-a065-4517-b44a-82c38c520f6b","Type":"ContainerStarted","Data":"59545ac6ae0ca2d42899d7aa9d88a5eeebeecb4a56d51a4e75fa79a7e57a02d9"} Jun 14 19:11:35.998407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:35.998370 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4fq4k" podStartSLOduration=5.235300066 podStartE2EDuration="23.998354155s" totalImagesPullingTime="18.763054089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:35.998295581 +0000 UTC m=+23.598245380" watchObservedRunningTime="2026-06-14 19:11:35.998354155 +0000 UTC m=+23.598303937" Jun 14 19:11:36.847553 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:36.847524 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:36.847718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:36.847556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:36.847718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:36.847556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:36.847718 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:36.847674 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:36.847886 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:36.847758 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:36.847886 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:36.847843 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:36.991559 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:36.991489 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" event={"ID":"feee0d5e-a065-4517-b44a-82c38c520f6b","Type":"ContainerStarted","Data":"febf049e3cd9e114b25565208ff5b28d7e9a8fbb9e37406d61ee97b61782e4a3"} Jun 14 19:11:37.008621 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.008567 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-z2nld" podStartSLOduration=3.2197443 podStartE2EDuration="24.008551764s" totalImagesPullingTime="20.788807464s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:37.008406418 +0000 UTC m=+24.608356203" watchObservedRunningTime="2026-06-14 19:11:37.008551764 +0000 UTC m=+24.608501545" Jun 14 19:11:37.966723 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.966650 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:37.967309 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.967288 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:37.997996 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.997964 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"8c76bc5ee00c34f39529771389727a012abb78a7c3e4c819a6d6461500ac5b30"} Jun 14 19:11:37.998358 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.998333 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:37.998618 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:37.998601 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-qgnw7" Jun 14 19:11:38.847195 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:38.847162 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:38.847363 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:38.847265 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:38.847363 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:38.847323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:38.847486 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:38.847403 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:38.847540 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:38.847501 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:38.847614 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:38.847596 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:40.006093 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.005817 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" event={"ID":"6d3cbe39-23e9-402d-b16f-b19dac8d1e1b","Type":"ContainerStarted","Data":"61b7d22f86ec38fccd24091560288d00ecae4f41a8ba87d08ccfe432f91d0f9a"} Jun 14 19:11:40.006860 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.006584 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:40.007075 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.007034 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:40.010806 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.010783 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="018bed29db18b39bb2e98c97ed727dbaaf02c985a0cdb057cd454c3336a1f9a7" exitCode=0 Jun 14 19:11:40.010903 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.010825 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"018bed29db18b39bb2e98c97ed727dbaaf02c985a0cdb057cd454c3336a1f9a7"} Jun 14 19:11:40.023222 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.023202 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:40.024148 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.024132 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:40.032242 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.032193 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" podStartSLOduration=8.879361936 podStartE2EDuration="28.032182623s" totalImagesPullingTime="19.152820687s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:40.031603551 +0000 UTC m=+27.631553342" watchObservedRunningTime="2026-06-14 19:11:40.032182623 +0000 UTC m=+27.632132405" Jun 14 19:11:40.847654 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.847608 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:40.847654 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.847623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:40.847860 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:40.847752 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:40.847860 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:40.847771 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:40.847929 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:40.847869 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:40.847962 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:40.847944 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:41.013194 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.013158 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 14 19:11:41.725875 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.725788 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nr9nh"] Jun 14 19:11:41.726066 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.725915 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:41.726066 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:41.726002 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:41.728899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.727414 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fhjvt"] Jun 14 19:11:41.728899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.727556 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:41.728899 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:41.727697 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:41.729860 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.729837 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zd6p"] Jun 14 19:11:41.729951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:41.729926 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:41.730071 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:41.730044 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:42.016995 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:42.016829 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="f5ba438ab3c3965030a789df45755394ecc69c77fa3d644c66c67b1d2d84571b" exitCode=0 Jun 14 19:11:42.017352 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:42.016920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"f5ba438ab3c3965030a789df45755394ecc69c77fa3d644c66c67b1d2d84571b"} Jun 14 19:11:42.017352 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:42.017102 2567 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 14 19:11:42.848466 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:42.848436 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:42.848621 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:42.848518 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:43.297364 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:43.297329 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:11:43.847509 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:43.847484 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:43.847614 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:43.847578 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:43.847674 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:43.847627 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:43.847747 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:43.847730 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:44.023468 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:44.023436 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="03507efccfaea6fad96c33a85a37b3b32bf656dd4c79a37f6cf1105c72d906bd" exitCode=0 Jun 14 19:11:44.023604 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:44.023478 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"03507efccfaea6fad96c33a85a37b3b32bf656dd4c79a37f6cf1105c72d906bd"} Jun 14 19:11:44.847963 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:44.847932 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:44.848554 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:44.848061 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:45.847065 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:45.847032 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:45.847244 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:45.847037 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:45.847244 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:45.847156 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fhjvt" podUID="fb06d62d-3d1b-45a9-88df-9e97b700a799" Jun 14 19:11:45.847329 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:45.847254 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:11:46.548826 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:46.548779 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:46.549311 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.548932 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:46.549311 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.549009 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:12:18.548989089 +0000 UTC m=+66.148938865 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 14 19:11:46.649546 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:46.649508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:46.649571 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649657 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649709 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649730 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649734 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret podName:ca26259f-d764-4620-bf6d-be60ee15e22e nodeName:}" failed. No retries permitted until 2026-06-14 19:12:18.649713315 +0000 UTC m=+66.249663079 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret") pod "global-pull-secret-syncer-9zd6p" (UID: "ca26259f-d764-4620-bf6d-be60ee15e22e") : object "kube-system"/"original-pull-secret" not registered Jun 14 19:11:46.649741 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649745 2567 projected.go:196] Error preparing data for projected volume kube-api-access-bhwng for pod openshift-network-diagnostics/network-check-target-fhjvt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:46.650034 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.649795 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng podName:fb06d62d-3d1b-45a9-88df-9e97b700a799 nodeName:}" failed. No retries permitted until 2026-06-14 19:12:18.649779135 +0000 UTC m=+66.249728899 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bhwng" (UniqueName: "kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng") pod "network-check-target-fhjvt" (UID: "fb06d62d-3d1b-45a9-88df-9e97b700a799") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 14 19:11:46.847446 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:46.847415 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:46.847628 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:46.847549 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9zd6p" podUID="ca26259f-d764-4620-bf6d-be60ee15e22e" Jun 14 19:11:47.184093 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.184020 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-135-103.ec2.internal" event="NodeReady" Jun 14 19:11:47.184230 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.184160 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 14 19:11:47.232557 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.232496 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-q9m4c"] Jun 14 19:11:47.235118 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.235093 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4p64b"] Jun 14 19:11:47.235255 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.235208 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.237280 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.237257 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.238243 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.238171 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 14 19:11:47.238243 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.238236 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 14 19:11:47.238418 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.238265 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nf5nn\"" Jun 14 19:11:47.239794 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.239769 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 14 19:11:47.239898 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.239810 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 14 19:11:47.239963 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.239899 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kbtq2\"" Jun 14 19:11:47.240259 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.240243 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 14 19:11:47.244513 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.244492 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p64b"] Jun 14 19:11:47.248396 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.248346 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9m4c"] Jun 14 19:11:47.261609 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.261583 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:47.355616 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqn97\" (UniqueName: \"kubernetes.io/projected/915d84aa-5f57-4116-955f-f02b9743be96-kube-api-access-lqn97\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.355803 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355626 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/915d84aa-5f57-4116-955f-f02b9743be96-config-volume\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.355803 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355694 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.355803 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355747 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0be6651-035f-4092-935e-a0d95c666a67-cert\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.355803 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355768 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtx22\" (UniqueName: \"kubernetes.io/projected/e0be6651-035f-4092-935e-a0d95c666a67-kube-api-access-wtx22\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.355986 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.355843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915d84aa-5f57-4116-955f-f02b9743be96-tmp-dir\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.360051 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.360023 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:47.409755 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.409720 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:47.456886 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456804 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqn97\" (UniqueName: \"kubernetes.io/projected/915d84aa-5f57-4116-955f-f02b9743be96-kube-api-access-lqn97\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.456886 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456855 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/915d84aa-5f57-4116-955f-f02b9743be96-config-volume\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.456886 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456880 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.457149 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456927 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0be6651-035f-4092-935e-a0d95c666a67-cert\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.457149 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456955 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtx22\" (UniqueName: \"kubernetes.io/projected/e0be6651-035f-4092-935e-a0d95c666a67-kube-api-access-wtx22\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.457149 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:47.456996 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:11:47.457149 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:47.457059 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:47.957039097 +0000 UTC m=+35.556988860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:11:47.457293 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.456999 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915d84aa-5f57-4116-955f-f02b9743be96-tmp-dir\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.457341 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.457321 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/915d84aa-5f57-4116-955f-f02b9743be96-tmp-dir\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.457675 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.457650 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/915d84aa-5f57-4116-955f-f02b9743be96-config-volume\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.461792 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.461772 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e0be6651-035f-4092-935e-a0d95c666a67-cert\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.465489 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.465463 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqn97\" (UniqueName: \"kubernetes.io/projected/915d84aa-5f57-4116-955f-f02b9743be96-kube-api-access-lqn97\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.466316 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.466296 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtx22\" (UniqueName: \"kubernetes.io/projected/e0be6651-035f-4092-935e-a0d95c666a67-kube-api-access-wtx22\") pod \"ingress-canary-4p64b\" (UID: \"e0be6651-035f-4092-935e-a0d95c666a67\") " pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.555455 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.555423 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4p64b" Jun 14 19:11:47.847432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.847407 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:11:47.847533 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.847444 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:11:47.850433 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.850410 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 14 19:11:47.850545 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.850444 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 14 19:11:47.850545 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.850447 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m44tw\"" Jun 14 19:11:47.851653 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.851617 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pxrhf\"" Jun 14 19:11:47.851653 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.851625 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 14 19:11:47.861874 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.861792 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4p64b"] Jun 14 19:11:47.867061 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:11:47.867034 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0be6651_035f_4092_935e_a0d95c666a67.slice/crio-603e8fc1e3ecdfb5e647db04f96b73215d41370781bf4b806da8f641c55e209c WatchSource:0}: Error finding container 603e8fc1e3ecdfb5e647db04f96b73215d41370781bf4b806da8f641c55e209c: Status 404 returned error can't find the container with id 603e8fc1e3ecdfb5e647db04f96b73215d41370781bf4b806da8f641c55e209c Jun 14 19:11:47.961150 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:47.960615 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:47.961150 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:47.960779 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:11:47.961150 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:47.960845 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:48.960825191 +0000 UTC m=+36.560774952 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:11:48.034504 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.034414 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p64b" event={"ID":"e0be6651-035f-4092-935e-a0d95c666a67","Type":"ContainerStarted","Data":"603e8fc1e3ecdfb5e647db04f96b73215d41370781bf4b806da8f641c55e209c"} Jun 14 19:11:48.037068 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.037044 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="4cb8f4c30ae870904b51b586b447957f517f637cc9172b73a9edf75cf3a477f2" exitCode=0 Jun 14 19:11:48.037159 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.037090 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"4cb8f4c30ae870904b51b586b447957f517f637cc9172b73a9edf75cf3a477f2"} Jun 14 19:11:48.847526 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.847495 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:11:48.850311 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.850285 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 14 19:11:48.968894 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:48.968853 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:48.969045 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:48.969007 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:11:48.969097 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:48.969075 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:50.969060441 +0000 UTC m=+38.569010205 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:11:49.043647 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:49.043589 2567 generic.go:356] "Generic (PLEG): container finished" podID="2ee89d03-f544-4b2f-8626-00b40bae5919" containerID="721d5f714efc18cd0e8543c419d4195a94acef0a55a9e3b4386f683174a3c102" exitCode=0 Jun 14 19:11:49.043815 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:49.043658 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerDied","Data":"721d5f714efc18cd0e8543c419d4195a94acef0a55a9e3b4386f683174a3c102"} Jun 14 19:11:49.882488 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:49.882308 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:49.929075 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:49.929047 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:50.047570 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.047537 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4p64b" event={"ID":"e0be6651-035f-4092-935e-a0d95c666a67","Type":"ContainerStarted","Data":"554425ad1fbf53159f8b6968d930f2d9e7b9b8f5e11fcc4998869addb318b0c1"} Jun 14 19:11:50.050675 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.050653 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-pwt78" event={"ID":"2ee89d03-f544-4b2f-8626-00b40bae5919","Type":"ContainerStarted","Data":"94022c6d203a9e6a48fa536e6c644470dbc4718938cc73bb9a67a8dd98f68c66"} Jun 14 19:11:50.062526 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.062484 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4p64b" podStartSLOduration=1.582101594 podStartE2EDuration="3.062468364s" totalImagesPullingTime="1.48036677s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:50.062003792 +0000 UTC m=+37.661953585" watchObservedRunningTime="2026-06-14 19:11:50.062468364 +0000 UTC m=+37.662418146" Jun 14 19:11:50.084615 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.084568 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-pwt78" podStartSLOduration=8.203113519 podStartE2EDuration="37.084554727s" totalImagesPullingTime="28.881441208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:11:50.082598489 +0000 UTC m=+37.682548270" watchObservedRunningTime="2026-06-14 19:11:50.084554727 +0000 UTC m=+37.684504508" Jun 14 19:11:50.201754 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.201679 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:50.302981 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.302943 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:50.413398 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.413327 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:50.987057 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:50.987021 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:50.987565 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:50.987155 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:11:50.987565 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:50.987217 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:11:54.987201142 +0000 UTC m=+42.587150900 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:11:52.709172 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:52.709118 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:52.766256 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:52.766217 2567 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" limit=1024 Jun 14 19:11:55.011679 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:11:55.011626 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:11:55.012028 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:55.011732 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:11:55.012028 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:11:55.011791 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:12:03.011775103 +0000 UTC m=+50.611724862 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:12:03.062262 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:03.062226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:12:03.062837 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:03.062353 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:12:03.062837 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:03.062416 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:12:19.062403188 +0000 UTC m=+66.662352947 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:12:13.307403 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:13.307375 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-lgwdf" Jun 14 19:12:18.573612 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.573581 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:12:18.576299 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.576281 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 14 19:12:18.584544 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:18.584530 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 14 19:12:18.584603 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:18.584594 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:13:22.584578684 +0000 UTC m=+130.184528444 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : secret "metrics-daemon-secret" not found Jun 14 19:12:18.673964 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.673937 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:12:18.674058 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.673978 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:12:18.676770 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.676755 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 14 19:12:18.676826 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.676791 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 14 19:12:18.686858 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.686842 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 14 19:12:18.688428 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.688411 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ca26259f-d764-4620-bf6d-be60ee15e22e-original-pull-secret\") pod \"global-pull-secret-syncer-9zd6p\" (UID: \"ca26259f-d764-4620-bf6d-be60ee15e22e\") " pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:12:18.697811 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.697787 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bhwng\" (UniqueName: \"kubernetes.io/projected/fb06d62d-3d1b-45a9-88df-9e97b700a799-kube-api-access-bhwng\") pod \"network-check-target-fhjvt\" (UID: \"fb06d62d-3d1b-45a9-88df-9e97b700a799\") " pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:12:18.776317 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.776296 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-pxrhf\"" Jun 14 19:12:18.784497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.784481 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:12:18.857997 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.857970 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9zd6p" Jun 14 19:12:18.945746 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.945705 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fhjvt"] Jun 14 19:12:18.948840 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:12:18.948811 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb06d62d_3d1b_45a9_88df_9e97b700a799.slice/crio-c31f6ed8a71b1a395be7dba2dc69243d579b6e1a4abd094b187e7497532e8ef3 WatchSource:0}: Error finding container c31f6ed8a71b1a395be7dba2dc69243d579b6e1a4abd094b187e7497532e8ef3: Status 404 returned error can't find the container with id c31f6ed8a71b1a395be7dba2dc69243d579b6e1a4abd094b187e7497532e8ef3 Jun 14 19:12:18.992744 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:18.992718 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9zd6p"] Jun 14 19:12:18.995431 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:12:18.995411 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca26259f_d764_4620_bf6d_be60ee15e22e.slice/crio-6a8c6c910cd9cf214c3b8fae248d09419f9064571d7ef80bb9bcf4a31bde1e0b WatchSource:0}: Error finding container 6a8c6c910cd9cf214c3b8fae248d09419f9064571d7ef80bb9bcf4a31bde1e0b: Status 404 returned error can't find the container with id 6a8c6c910cd9cf214c3b8fae248d09419f9064571d7ef80bb9bcf4a31bde1e0b Jun 14 19:12:19.077733 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:19.077683 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:12:19.077824 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:19.077811 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:12:19.077866 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:19.077859 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:12:51.077844934 +0000 UTC m=+98.677794693 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:12:19.114597 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:19.114568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9zd6p" event={"ID":"ca26259f-d764-4620-bf6d-be60ee15e22e","Type":"ContainerStarted","Data":"6a8c6c910cd9cf214c3b8fae248d09419f9064571d7ef80bb9bcf4a31bde1e0b"} Jun 14 19:12:19.115569 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:19.115539 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fhjvt" event={"ID":"fb06d62d-3d1b-45a9-88df-9e97b700a799","Type":"ContainerStarted","Data":"c31f6ed8a71b1a395be7dba2dc69243d579b6e1a4abd094b187e7497532e8ef3"} Jun 14 19:12:24.128935 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:24.128891 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9zd6p" event={"ID":"ca26259f-d764-4620-bf6d-be60ee15e22e","Type":"ContainerStarted","Data":"085d78bc2c3a0dade2149956b0be34875bd124e8a2ae66bb4faeaca0ecc7c146"} Jun 14 19:12:24.130326 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:24.130300 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fhjvt" event={"ID":"fb06d62d-3d1b-45a9-88df-9e97b700a799","Type":"ContainerStarted","Data":"d669a63a85e4522d2b62dead71d184f850ba753d2141a7a44c42355667419608"} Jun 14 19:12:24.130456 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:24.130444 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:12:24.143678 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:24.143623 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9zd6p" podStartSLOduration=66.977532744 podStartE2EDuration="1m11.143605904s" totalImagesPullingTime="4.16607316s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:12:24.14352221 +0000 UTC m=+71.743471990" watchObservedRunningTime="2026-06-14 19:12:24.143605904 +0000 UTC m=+71.743555669" Jun 14 19:12:24.161733 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:24.161692 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fhjvt" podStartSLOduration=66.953559977 podStartE2EDuration="1m11.161678876s" totalImagesPullingTime="4.208118899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:11:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:12:24.160898988 +0000 UTC m=+71.760848769" watchObservedRunningTime="2026-06-14 19:12:24.161678876 +0000 UTC m=+71.761628658" Jun 14 19:12:36.545464 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.545346 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85"] Jun 14 19:12:36.547905 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.547886 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.548444 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.548422 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt"] Jun 14 19:12:36.550246 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.550228 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.551738 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.551714 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 14 19:12:36.551738 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.551733 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 14 19:12:36.551893 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.551716 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-sps97\"" Jun 14 19:12:36.551893 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.551720 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 14 19:12:36.551893 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.551716 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 14 19:12:36.553504 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.553488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 14 19:12:36.553664 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.553574 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 14 19:12:36.553738 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.553727 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 14 19:12:36.555718 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.555697 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 14 19:12:36.557815 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.557797 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85"] Jun 14 19:12:36.564299 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.564276 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt"] Jun 14 19:12:36.591329 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591306 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.591425 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591357 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2jjf\" (UniqueName: \"kubernetes.io/projected/be358f2e-8b1b-45b3-b28f-868cc84a624f-kube-api-access-q2jjf\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.591425 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dsz5\" (UniqueName: \"kubernetes.io/projected/48f16c35-625b-49a3-975c-47d02710a812-kube-api-access-8dsz5\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.591510 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591433 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.591510 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591474 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be358f2e-8b1b-45b3-b28f-868cc84a624f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.591510 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591502 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f16c35-625b-49a3-975c-47d02710a812-additional-service-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.591616 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591586 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.591672 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.591612 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692579 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692548 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2jjf\" (UniqueName: \"kubernetes.io/projected/be358f2e-8b1b-45b3-b28f-868cc84a624f-kube-api-access-q2jjf\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.692741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692650 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dsz5\" (UniqueName: \"kubernetes.io/projected/48f16c35-625b-49a3-975c-47d02710a812-kube-api-access-8dsz5\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692677 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692713 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be358f2e-8b1b-45b3-b28f-868cc84a624f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.692956 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692826 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f16c35-625b-49a3-975c-47d02710a812-additional-service-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692956 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.692956 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.692909 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.693477 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.693456 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/48f16c35-625b-49a3-975c-47d02710a812-additional-service-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.695149 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.695129 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.695577 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.695558 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-ca\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.695675 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.695655 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.695720 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.695661 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be358f2e-8b1b-45b3-b28f-868cc84a624f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.696043 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.696026 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/48f16c35-625b-49a3-975c-47d02710a812-hub\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.700477 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.700446 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2jjf\" (UniqueName: \"kubernetes.io/projected/be358f2e-8b1b-45b3-b28f-868cc84a624f-kube-api-access-q2jjf\") pod \"managed-serviceaccount-addon-agent-5fd7464686-jpw85\" (UID: \"be358f2e-8b1b-45b3-b28f-868cc84a624f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.700723 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.700703 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dsz5\" (UniqueName: \"kubernetes.io/projected/48f16c35-625b-49a3-975c-47d02710a812-kube-api-access-8dsz5\") pod \"cluster-proxy-proxy-agent-69479c9974-gg8nt\" (UID: \"48f16c35-625b-49a3-975c-47d02710a812\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.865739 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.865717 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" Jun 14 19:12:36.870966 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.870942 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:12:36.995385 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:36.995355 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85"] Jun 14 19:12:36.998393 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:12:36.998368 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbe358f2e_8b1b_45b3_b28f_868cc84a624f.slice/crio-ddd4538e7643ea894a514d648acc8c0f53da3e046a412bec045c874f9f435c5c WatchSource:0}: Error finding container ddd4538e7643ea894a514d648acc8c0f53da3e046a412bec045c874f9f435c5c: Status 404 returned error can't find the container with id ddd4538e7643ea894a514d648acc8c0f53da3e046a412bec045c874f9f435c5c Jun 14 19:12:37.010561 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:37.010494 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt"] Jun 14 19:12:37.012807 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:12:37.012787 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48f16c35_625b_49a3_975c_47d02710a812.slice/crio-3d46d63ec7b7367f0ddf5f648f67944493a0bd929daf0e14d4a36fe9391a7b89 WatchSource:0}: Error finding container 3d46d63ec7b7367f0ddf5f648f67944493a0bd929daf0e14d4a36fe9391a7b89: Status 404 returned error can't find the container with id 3d46d63ec7b7367f0ddf5f648f67944493a0bd929daf0e14d4a36fe9391a7b89 Jun 14 19:12:37.159855 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:37.159788 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" event={"ID":"be358f2e-8b1b-45b3-b28f-868cc84a624f","Type":"ContainerStarted","Data":"ddd4538e7643ea894a514d648acc8c0f53da3e046a412bec045c874f9f435c5c"} Jun 14 19:12:37.160780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:37.160756 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerStarted","Data":"3d46d63ec7b7367f0ddf5f648f67944493a0bd929daf0e14d4a36fe9391a7b89"} Jun 14 19:12:41.179941 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.179898 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerStarted","Data":"fd9e2b69aeec023a78787465d200ec149533acb8bd9c5a69a8f1a90650af6287"} Jun 14 19:12:41.180417 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.179950 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerStarted","Data":"7ea1282ecb937fed5b151314931e5e8dc79e2dd6eaeb2b53c582c6fa02020e5b"} Jun 14 19:12:41.180417 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.179969 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerStarted","Data":"e04556c96c735b8beb5910f17acfa1d88e08d36458dfc91960a1acdcccb4947a"} Jun 14 19:12:41.181291 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.181267 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" event={"ID":"be358f2e-8b1b-45b3-b28f-868cc84a624f","Type":"ContainerStarted","Data":"663f48d0ee478084975662847d6d4ca89581133246d96e8596f02b298cdb07dd"} Jun 14 19:12:41.198302 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.198260 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" podStartSLOduration=1.438040414 podStartE2EDuration="5.198246226s" totalImagesPullingTime="3.760205812s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:12:41.196439329 +0000 UTC m=+88.796389110" watchObservedRunningTime="2026-06-14 19:12:41.198246226 +0000 UTC m=+88.798196006" Jun 14 19:12:41.210954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:41.210913 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" podStartSLOduration=1.4460629 podStartE2EDuration="5.210900048s" totalImagesPullingTime="3.764837148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:12:41.210346614 +0000 UTC m=+88.810296400" watchObservedRunningTime="2026-06-14 19:12:41.210900048 +0000 UTC m=+88.810849854" Jun 14 19:12:51.093153 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:51.093117 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:12:51.093556 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:51.093226 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:12:51.093556 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:12:51.093283 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:13:55.093271536 +0000 UTC m=+162.693221295 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:12:55.135798 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:12:55.135758 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fhjvt" Jun 14 19:13:22.594002 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:13:22.593958 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:13:22.594396 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:22.594085 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 14 19:13:22.594396 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:22.594146 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs podName:3f5263ee-a22a-4417-9d14-ff9f08c65b1a nodeName:}" failed. No retries permitted until 2026-06-14 19:15:24.594129577 +0000 UTC m=+252.194079337 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs") pod "network-metrics-daemon-nr9nh" (UID: "3f5263ee-a22a-4417-9d14-ff9f08c65b1a") : secret "metrics-daemon-secret" not found Jun 14 19:13:50.249326 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:50.249284 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9m4c" podUID="915d84aa-5f57-4116-955f-f02b9743be96" Jun 14 19:13:50.373900 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:13:50.373869 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:13:50.868010 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:50.867973 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-nr9nh" podUID="3f5263ee-a22a-4417-9d14-ff9f08c65b1a" Jun 14 19:13:55.109318 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:13:55.109285 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:13:55.109704 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:55.109403 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 14 19:13:55.109704 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:13:55.109463 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls podName:915d84aa-5f57-4116-955f-f02b9743be96 nodeName:}" failed. No retries permitted until 2026-06-14 19:15:57.109448342 +0000 UTC m=+284.709398102 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls") pod "dns-default-q9m4c" (UID: "915d84aa-5f57-4116-955f-f02b9743be96") : secret "dns-default-metrics-tls" not found Jun 14 19:14:03.847570 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:03.847472 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:14:41.518135 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:41.518102 2567 generic.go:356] "Generic (PLEG): container finished" podID="be358f2e-8b1b-45b3-b28f-868cc84a624f" containerID="663f48d0ee478084975662847d6d4ca89581133246d96e8596f02b298cdb07dd" exitCode=255 Jun 14 19:14:41.518520 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:41.518176 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" event={"ID":"be358f2e-8b1b-45b3-b28f-868cc84a624f","Type":"ContainerDied","Data":"663f48d0ee478084975662847d6d4ca89581133246d96e8596f02b298cdb07dd"} Jun 14 19:14:41.518520 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:41.518425 2567 scope.go:117] "RemoveContainer" containerID="663f48d0ee478084975662847d6d4ca89581133246d96e8596f02b298cdb07dd" Jun 14 19:14:42.522959 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:42.522927 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5fd7464686-jpw85" event={"ID":"be358f2e-8b1b-45b3-b28f-868cc84a624f","Type":"ContainerStarted","Data":"5e160782353ddbd424bf938f72fb231ea8d90923bd6f2a2e110722e2dc1b6c3b"} Jun 14 19:14:50.807335 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.807294 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-k4p54"] Jun 14 19:14:50.810355 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.810341 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.813028 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.813007 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-qjxw9\"" Jun 14 19:14:50.814228 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.814206 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 14 19:14:50.814338 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.814206 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 14 19:14:50.814338 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.814227 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 14 19:14:50.814338 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.814227 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 14 19:14:50.820531 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.820499 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k4p54"] Jun 14 19:14:50.882208 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.882182 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-data-volume\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.882337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.882220 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-crio-socket\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.882337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.882243 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.882337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.882262 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.882547 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.882332 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rkgv\" (UniqueName: \"kubernetes.io/projected/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-api-access-5rkgv\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983020 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.982989 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-crio-socket\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983020 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rkgv\" (UniqueName: \"kubernetes.io/projected/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-api-access-5rkgv\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983116 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-crio-socket\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:50.983144 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:50.983178 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:50.983217 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:51.483200828 +0000 UTC m=+219.083150587 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : configmap "kube-rbac-proxy" not found Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983130 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-data-volume\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.983241 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:50.983247 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:51.483229622 +0000 UTC m=+219.083179396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : secret "insights-runtime-extractor-tls" not found Jun 14 19:14:50.983574 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.983373 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-data-volume\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:50.990736 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:50.990718 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rkgv\" (UniqueName: \"kubernetes.io/projected/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-api-access-5rkgv\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:51.486415 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:51.486363 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:51.486415 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:51.486421 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:51.486690 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:51.486492 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 14 19:14:51.486690 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:51.486560 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 14 19:14:51.486690 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:51.486565 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:52.486548776 +0000 UTC m=+220.086498539 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : configmap "kube-rbac-proxy" not found Jun 14 19:14:51.486690 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:51.486627 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:52.486611903 +0000 UTC m=+220.086561662 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : secret "insights-runtime-extractor-tls" not found Jun 14 19:14:52.493680 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:52.493647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:52.493680 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:52.493681 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:52.494190 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:52.493795 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 14 19:14:52.494190 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:52.493858 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 14 19:14:52.494190 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:52.493875 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:54.493855468 +0000 UTC m=+222.093805243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : configmap "kube-rbac-proxy" not found Jun 14 19:14:52.494190 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:52.493914 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:54.493901585 +0000 UTC m=+222.093851348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : secret "insights-runtime-extractor-tls" not found Jun 14 19:14:52.584994 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:52.584970 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-695qr_51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e/dns-node-resolver/0.log" Jun 14 19:14:53.786313 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:53.786280 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6m4m_5655dd8a-5500-41b3-add1-0dc28a18292f/node-ca/0.log" Jun 14 19:14:54.385677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:54.385630 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p64b_e0be6651-035f-4092-935e-a0d95c666a67/serve-healthcheck-canary/0.log" Jun 14 19:14:54.508757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:54.508719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:54.508958 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:54.508834 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:54.508958 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:54.508870 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 14 19:14:54.508958 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:54.508908 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 14 19:14:54.508958 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:54.508929 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:58.508914993 +0000 UTC m=+226.108864756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : secret "insights-runtime-extractor-tls" not found Jun 14 19:14:54.508958 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:54.508954 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:14:58.508940159 +0000 UTC m=+226.108889922 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : configmap "kube-rbac-proxy" not found Jun 14 19:14:58.540864 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:58.540816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:58.540864 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:14:58.540870 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:14:58.541326 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:58.540931 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 14 19:14:58.541326 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:58.540990 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:15:06.540976619 +0000 UTC m=+234.140926378 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : configmap "kube-rbac-proxy" not found Jun 14 19:14:58.541326 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:58.541022 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 14 19:14:58.541326 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:14:58.541108 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls podName:43f5796f-b020-46d7-97b3-1f3ef75ffb0a nodeName:}" failed. No retries permitted until 2026-06-14 19:15:06.541092829 +0000 UTC m=+234.141042600 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-k4p54" (UID: "43f5796f-b020-46d7-97b3-1f3ef75ffb0a") : secret "insights-runtime-extractor-tls" not found Jun 14 19:15:06.603337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.603300 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:15:06.603337 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.603339 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:15:06.603846 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.603820 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:15:06.605532 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.605517 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/43f5796f-b020-46d7-97b3-1f3ef75ffb0a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-k4p54\" (UID: \"43f5796f-b020-46d7-97b3-1f3ef75ffb0a\") " pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:15:06.719252 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.719216 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-k4p54" Jun 14 19:15:06.846651 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:06.846600 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-k4p54"] Jun 14 19:15:06.850128 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:06.850080 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43f5796f_b020_46d7_97b3_1f3ef75ffb0a.slice/crio-76446b5fbddf0ca08709c5f70a3a10230aaffa338da2afb88cadebdd3d84e6f3 WatchSource:0}: Error finding container 76446b5fbddf0ca08709c5f70a3a10230aaffa338da2afb88cadebdd3d84e6f3: Status 404 returned error can't find the container with id 76446b5fbddf0ca08709c5f70a3a10230aaffa338da2afb88cadebdd3d84e6f3 Jun 14 19:15:07.596100 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:07.596021 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k4p54" event={"ID":"43f5796f-b020-46d7-97b3-1f3ef75ffb0a","Type":"ContainerStarted","Data":"76446b5fbddf0ca08709c5f70a3a10230aaffa338da2afb88cadebdd3d84e6f3"} Jun 14 19:15:10.606373 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:10.606337 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k4p54" event={"ID":"43f5796f-b020-46d7-97b3-1f3ef75ffb0a","Type":"ContainerStarted","Data":"c6e176cc43c13dd0286bf12a0d60380681d637bc0f24324e5d771c4e5fbbea2d"} Jun 14 19:15:11.611754 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.611713 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k4p54" event={"ID":"43f5796f-b020-46d7-97b3-1f3ef75ffb0a","Type":"ContainerStarted","Data":"b79bffefadcd512ed89d153fb5a4017914e60ed1f590c835116ed420c833ef54"} Jun 14 19:15:11.661389 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.661360 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j"] Jun 14 19:15:11.664457 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.664428 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.667202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.667178 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 14 19:15:11.667386 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.667199 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 14 19:15:11.667386 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.667353 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-6qht7\"" Jun 14 19:15:11.667542 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.667437 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 14 19:15:11.673493 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.673473 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 14 19:15:11.677170 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.677141 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j"] Jun 14 19:15:11.745513 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745476 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-image-registry-private-configuration\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745660 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745518 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-bound-sa-token\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745660 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-trusted-ca\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745660 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ad0393-407d-438f-b98b-70758d12d005-ca-trust-extracted\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745790 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-installation-pull-secrets\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745845 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ds6b6\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-kube-api-access-ds6b6\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745845 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745831 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-registry-certificates\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.745912 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.745857 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-registry-tls\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847088 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ds6b6\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-kube-api-access-ds6b6\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847097 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-registry-certificates\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847160 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-registry-tls\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847204 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-image-registry-private-configuration\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-bound-sa-token\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847257 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847251 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-trusted-ca\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847557 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847430 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ad0393-407d-438f-b98b-70758d12d005-ca-trust-extracted\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847557 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-installation-pull-secrets\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.847888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.847859 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/f2ad0393-407d-438f-b98b-70758d12d005-ca-trust-extracted\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.848093 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.848044 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-registry-certificates\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.848211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.848151 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/f2ad0393-407d-438f-b98b-70758d12d005-trusted-ca\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.849980 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.849957 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-installation-pull-secrets\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.850224 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.850202 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/f2ad0393-407d-438f-b98b-70758d12d005-image-registry-private-configuration\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.850305 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.850289 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-registry-tls\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.857206 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.857180 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ds6b6\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-kube-api-access-ds6b6\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.857310 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.857275 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f2ad0393-407d-438f-b98b-70758d12d005-bound-sa-token\") pod \"image-registry-6c6fd6c8b4-zmz6j\" (UID: \"f2ad0393-407d-438f-b98b-70758d12d005\") " pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:11.975498 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:11.975426 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:12.487202 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.487177 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j"] Jun 14 19:15:12.489789 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:12.489768 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2ad0393_407d_438f_b98b_70758d12d005.slice/crio-659ba4f0fe438bab40c0a408744a2a022c95a045a345ad2719ddd911aff631c2 WatchSource:0}: Error finding container 659ba4f0fe438bab40c0a408744a2a022c95a045a345ad2719ddd911aff631c2: Status 404 returned error can't find the container with id 659ba4f0fe438bab40c0a408744a2a022c95a045a345ad2719ddd911aff631c2 Jun 14 19:15:12.616882 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.616850 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-k4p54" event={"ID":"43f5796f-b020-46d7-97b3-1f3ef75ffb0a","Type":"ContainerStarted","Data":"226fb68f3307ebdfff5c065bc8c74435caba46d66061d4810658edbb5c6ebbf4"} Jun 14 19:15:12.618158 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.618132 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" event={"ID":"f2ad0393-407d-438f-b98b-70758d12d005","Type":"ContainerStarted","Data":"56d52b6f86b4cce9b411b22209c8090c911435b6edc27cd28ff62914a331ada4"} Jun 14 19:15:12.618271 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.618182 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:12.618271 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.618199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" event={"ID":"f2ad0393-407d-438f-b98b-70758d12d005","Type":"ContainerStarted","Data":"659ba4f0fe438bab40c0a408744a2a022c95a045a345ad2719ddd911aff631c2"} Jun 14 19:15:12.634587 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.634507 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-k4p54" podStartSLOduration=17.176558992 podStartE2EDuration="22.634493596s" totalImagesPullingTime="5.457934604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:14:50 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:12.634025464 +0000 UTC m=+240.233975247" watchObservedRunningTime="2026-06-14 19:15:12.634493596 +0000 UTC m=+240.234443402" Jun 14 19:15:12.656188 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:12.656145 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podStartSLOduration=1.656127275 podStartE2EDuration="1.656127275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:15:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:12.654843354 +0000 UTC m=+240.254793157" watchObservedRunningTime="2026-06-14 19:15:12.656127275 +0000 UTC m=+240.256077057" Jun 14 19:15:16.872867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:16.872824 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" podUID="48f16c35-625b-49a3-975c-47d02710a812" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 14 19:15:24.639231 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:24.639169 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:15:24.641523 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:24.641495 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3f5263ee-a22a-4417-9d14-ff9f08c65b1a-metrics-certs\") pod \"network-metrics-daemon-nr9nh\" (UID: \"3f5263ee-a22a-4417-9d14-ff9f08c65b1a\") " pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:15:24.851143 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:24.851113 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-m44tw\"" Jun 14 19:15:24.859246 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:24.859227 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-nr9nh" Jun 14 19:15:24.985179 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:24.985152 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-nr9nh"] Jun 14 19:15:24.988225 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:24.988196 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f5263ee_a22a_4417_9d14_ff9f08c65b1a.slice/crio-06cc6df8688285b17c346221c956c48877eb773644d209d6b1c0880594aa5668 WatchSource:0}: Error finding container 06cc6df8688285b17c346221c956c48877eb773644d209d6b1c0880594aa5668: Status 404 returned error can't find the container with id 06cc6df8688285b17c346221c956c48877eb773644d209d6b1c0880594aa5668 Jun 14 19:15:25.659871 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:25.659836 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nr9nh" event={"ID":"3f5263ee-a22a-4417-9d14-ff9f08c65b1a","Type":"ContainerStarted","Data":"06cc6df8688285b17c346221c956c48877eb773644d209d6b1c0880594aa5668"} Jun 14 19:15:26.664176 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:26.664141 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nr9nh" event={"ID":"3f5263ee-a22a-4417-9d14-ff9f08c65b1a","Type":"ContainerStarted","Data":"e6b7732cb718145a326947e46e5eefb28f686bddd245024aee99864a4f62edd3"} Jun 14 19:15:26.664544 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:26.664198 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-nr9nh" event={"ID":"3f5263ee-a22a-4417-9d14-ff9f08c65b1a","Type":"ContainerStarted","Data":"7eab67ad02cfa3fccba10950d1fdd3cdf9bc8f85ae1d095ac74af7f3c1f51a84"} Jun 14 19:15:26.679272 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:26.679227 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-nr9nh" podStartSLOduration=253.731023128 podStartE2EDuration="4m14.679214519s" totalImagesPullingTime="948.191391ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:11:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:26.677716277 +0000 UTC m=+254.277666060" watchObservedRunningTime="2026-06-14 19:15:26.679214519 +0000 UTC m=+254.279164300" Jun 14 19:15:26.872580 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:26.872550 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" podUID="48f16c35-625b-49a3-975c-47d02710a812" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 14 19:15:29.067630 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.067596 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc"] Jun 14 19:15:29.086716 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.086695 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc"] Jun 14 19:15:29.086869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.086786 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.089311 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.089289 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jun 14 19:15:29.089456 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.089391 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 14 19:15:29.091869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.091538 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 14 19:15:29.091869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.091596 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5cllb\"" Jun 14 19:15:29.091869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.091650 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jun 14 19:15:29.091869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.091809 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 14 19:15:29.092157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.091997 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-t48md"] Jun 14 19:15:29.108558 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.108532 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.110861 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.110844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 14 19:15:29.110861 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.110854 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 14 19:15:29.111016 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.110891 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-x97r6\"" Jun 14 19:15:29.111226 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.111210 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 14 19:15:29.163960 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.163939 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-root\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164077 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.163967 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kghlq\" (UniqueName: \"kubernetes.io/projected/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-kube-api-access-kghlq\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164077 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164023 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-tls\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164170 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164081 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164170 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164111 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-wtmp\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164170 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164125 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-metrics-client-ca\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164170 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164162 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-sys\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164347 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164222 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-textfile\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.164347 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.164258 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.264905 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.264848 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kghlq\" (UniqueName: \"kubernetes.io/projected/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-kube-api-access-kghlq\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.264905 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.264884 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-tls\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.264906 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.264931 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.265060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.264993 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jkpn4\" (UniqueName: \"kubernetes.io/projected/9e8b7536-57b4-4133-9e92-823910ebf461-kube-api-access-jkpn4\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.265060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265025 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-wtmp\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265048 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-metrics-client-ca\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265138 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8b7536-57b4-4133-9e92-823910ebf461-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265189 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-sys\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265193 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-wtmp\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265216 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265246 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-sys\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265267 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-textfile\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265490 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265490 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265314 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-root\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265490 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265374 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-root\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265591 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265572 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-textfile\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265591 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265579 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-metrics-client-ca\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.265685 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.265579 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.267200 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.267184 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-tls\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.267326 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.267310 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.272507 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.272477 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kghlq\" (UniqueName: \"kubernetes.io/projected/b7047cff-c0e7-4c86-acdd-81643ba4fd8f-kube-api-access-kghlq\") pod \"node-exporter-t48md\" (UID: \"b7047cff-c0e7-4c86-acdd-81643ba4fd8f\") " pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.365720 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.365617 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.365720 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.365703 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jkpn4\" (UniqueName: \"kubernetes.io/projected/9e8b7536-57b4-4133-9e92-823910ebf461-kube-api-access-jkpn4\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.365890 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.365745 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8b7536-57b4-4133-9e92-823910ebf461-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.365890 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.365798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.365969 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:15:29.365933 2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jun 14 19:15:29.366015 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:15:29.366005 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls podName:9e8b7536-57b4-4133-9e92-823910ebf461 nodeName:}" failed. No retries permitted until 2026-06-14 19:15:29.865987007 +0000 UTC m=+257.465936772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls") pod "openshift-state-metrics-6b8d965dd5-lk6vc" (UID: "9e8b7536-57b4-4133-9e92-823910ebf461") : secret "openshift-state-metrics-tls" not found Jun 14 19:15:29.366425 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.366397 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9e8b7536-57b4-4133-9e92-823910ebf461-metrics-client-ca\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.368006 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.367984 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.374105 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.374078 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jkpn4\" (UniqueName: \"kubernetes.io/projected/9e8b7536-57b4-4133-9e92-823910ebf461-kube-api-access-jkpn4\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.416460 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.416437 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-t48md" Jun 14 19:15:29.425446 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:29.425420 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7047cff_c0e7_4c86_acdd_81643ba4fd8f.slice/crio-bc79e7b5884e317ca778baf8e18bab4a08f68977e099e38dd667d2e0b9165175 WatchSource:0}: Error finding container bc79e7b5884e317ca778baf8e18bab4a08f68977e099e38dd667d2e0b9165175: Status 404 returned error can't find the container with id bc79e7b5884e317ca778baf8e18bab4a08f68977e099e38dd667d2e0b9165175 Jun 14 19:15:29.674370 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.674294 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t48md" event={"ID":"b7047cff-c0e7-4c86-acdd-81643ba4fd8f","Type":"ContainerStarted","Data":"bc79e7b5884e317ca778baf8e18bab4a08f68977e099e38dd667d2e0b9165175"} Jun 14 19:15:29.869242 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.869213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.871424 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.871399 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/9e8b7536-57b4-4133-9e92-823910ebf461-openshift-state-metrics-tls\") pod \"openshift-state-metrics-6b8d965dd5-lk6vc\" (UID: \"9e8b7536-57b4-4133-9e92-823910ebf461\") " pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:29.996999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:29.996930 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" Jun 14 19:15:30.135084 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.134878 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc"] Jun 14 19:15:30.137629 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:30.137600 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e8b7536_57b4_4133_9e92_823910ebf461.slice/crio-851ef5c5ed0fa4e90b1b5b2d5ad825796f6c1c0155b2c46f371cca9c306b8bde WatchSource:0}: Error finding container 851ef5c5ed0fa4e90b1b5b2d5ad825796f6c1c0155b2c46f371cca9c306b8bde: Status 404 returned error can't find the container with id 851ef5c5ed0fa4e90b1b5b2d5ad825796f6c1c0155b2c46f371cca9c306b8bde Jun 14 19:15:30.681051 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.680972 2567 generic.go:356] "Generic (PLEG): container finished" podID="b7047cff-c0e7-4c86-acdd-81643ba4fd8f" containerID="c32ca65be0b0fcc738c0446837f5da52ccda1d949aabff15a428127d59b0f1f9" exitCode=0 Jun 14 19:15:30.681051 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.681045 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t48md" event={"ID":"b7047cff-c0e7-4c86-acdd-81643ba4fd8f","Type":"ContainerDied","Data":"c32ca65be0b0fcc738c0446837f5da52ccda1d949aabff15a428127d59b0f1f9"} Jun 14 19:15:30.682885 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.682854 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" event={"ID":"9e8b7536-57b4-4133-9e92-823910ebf461","Type":"ContainerStarted","Data":"bed71934d378da83c5d4270efd8cdc22985d49d0213d65b7459b43b02c6e7896"} Jun 14 19:15:30.682991 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.682897 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" event={"ID":"9e8b7536-57b4-4133-9e92-823910ebf461","Type":"ContainerStarted","Data":"cfe5c0f88ceb60542ecabc67bd327103082ef4fba8a68689ebdd45ac13c3f85e"} Jun 14 19:15:30.682991 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:30.682915 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" event={"ID":"9e8b7536-57b4-4133-9e92-823910ebf461","Type":"ContainerStarted","Data":"851ef5c5ed0fa4e90b1b5b2d5ad825796f6c1c0155b2c46f371cca9c306b8bde"} Jun 14 19:15:31.687875 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.687775 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t48md" event={"ID":"b7047cff-c0e7-4c86-acdd-81643ba4fd8f","Type":"ContainerStarted","Data":"f41771207d09b0bf53aec2675cd86dc2a2fdfc8e2578c45b45959b324cbe125e"} Jun 14 19:15:31.687875 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.687836 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-t48md" event={"ID":"b7047cff-c0e7-4c86-acdd-81643ba4fd8f","Type":"ContainerStarted","Data":"7d37bbef26ed188c03a61c8cadb90ed6062d5ff6184d4f30358b6416ec882c7d"} Jun 14 19:15:31.690079 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.690057 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" event={"ID":"9e8b7536-57b4-4133-9e92-823910ebf461","Type":"ContainerStarted","Data":"bf3c943087fdcfef16ae1efcbd49296263f98063c640f29c04203266dfa786bb"} Jun 14 19:15:31.707157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.707117 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-t48md" podStartSLOduration=1.703215653 podStartE2EDuration="2.707103389s" totalImagesPullingTime="935.293721ms" totalInitContainerRuntime="68.594015ms" isStatefulPod=true podCreationTimestamp="2026-06-14 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:31.705579436 +0000 UTC m=+259.305529231" watchObservedRunningTime="2026-06-14 19:15:31.707103389 +0000 UTC m=+259.307053210" Jun 14 19:15:31.722022 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.721978 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-6b8d965dd5-lk6vc" podStartSLOduration=1.7772876819999999 podStartE2EDuration="2.721951161s" totalImagesPullingTime="944.663479ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:15:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:31.721536093 +0000 UTC m=+259.321485885" watchObservedRunningTime="2026-06-14 19:15:31.721951161 +0000 UTC m=+259.321900943" Jun 14 19:15:31.979695 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.979605 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:31.979695 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:31.979682 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:33.626329 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.626290 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:33.626799 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.626361 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:33.853385 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.853353 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:15:33.856481 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.856466 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.860517 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860490 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860522 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860527 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mxpzn\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860542 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860584 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860528 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 14 19:15:33.860692 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860529 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 14 19:15:33.861081 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.860777 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 14 19:15:33.865230 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.865207 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 14 19:15:33.867677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.867658 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:15:33.902122 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902071 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902122 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902103 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902133 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnv5g\" (UniqueName: \"kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902200 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902292 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902238 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902292 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902261 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:33.902292 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:33.902277 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.002771 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.002748 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.002865 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.002781 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.002865 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.002807 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnv5g\" (UniqueName: \"kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.002865 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.002842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.002980 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.002905 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003060 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003041 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003120 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003088 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003545 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003521 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003666 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003558 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003666 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003614 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.003753 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.003704 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.005726 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.005711 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.005913 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.005896 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.009942 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.009919 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnv5g\" (UniqueName: \"kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g\") pod \"console-5859b7696f-nhdpp\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.165776 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.165667 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:34.292516 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.292402 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:15:34.294933 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:34.294894 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0518e718_4ef5_49b6_8c1d_4e0dd32d35bb.slice/crio-70a86a4739c6f4556fbb39311d619a105063be9757e8be250f78d8c0684dd9c3 WatchSource:0}: Error finding container 70a86a4739c6f4556fbb39311d619a105063be9757e8be250f78d8c0684dd9c3: Status 404 returned error can't find the container with id 70a86a4739c6f4556fbb39311d619a105063be9757e8be250f78d8c0684dd9c3 Jun 14 19:15:34.704156 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:34.704116 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859b7696f-nhdpp" event={"ID":"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb","Type":"ContainerStarted","Data":"70a86a4739c6f4556fbb39311d619a105063be9757e8be250f78d8c0684dd9c3"} Jun 14 19:15:36.872592 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:36.872554 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" podUID="48f16c35-625b-49a3-975c-47d02710a812" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 14 19:15:36.873053 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:36.872623 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" Jun 14 19:15:36.873214 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:36.873180 2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"fd9e2b69aeec023a78787465d200ec149533acb8bd9c5a69a8f1a90650af6287"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 14 19:15:36.873291 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:36.873236 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" podUID="48f16c35-625b-49a3-975c-47d02710a812" containerName="service-proxy" containerID="cri-o://fd9e2b69aeec023a78787465d200ec149533acb8bd9c5a69a8f1a90650af6287" gracePeriod=30 Jun 14 19:15:37.717726 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:37.717676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859b7696f-nhdpp" event={"ID":"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb","Type":"ContainerStarted","Data":"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80"} Jun 14 19:15:37.719849 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:37.719818 2567 generic.go:356] "Generic (PLEG): container finished" podID="48f16c35-625b-49a3-975c-47d02710a812" containerID="fd9e2b69aeec023a78787465d200ec149533acb8bd9c5a69a8f1a90650af6287" exitCode=2 Jun 14 19:15:37.719978 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:37.719870 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerDied","Data":"fd9e2b69aeec023a78787465d200ec149533acb8bd9c5a69a8f1a90650af6287"} Jun 14 19:15:37.719978 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:37.719897 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-69479c9974-gg8nt" event={"ID":"48f16c35-625b-49a3-975c-47d02710a812","Type":"ContainerStarted","Data":"5893c3b56769a45f7f999f885135967213af741369a63820cb7747d2baffe120"} Jun 14 19:15:37.734291 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:37.734247 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5859b7696f-nhdpp" podStartSLOduration=2.236271394 podStartE2EDuration="4.734233657s" totalImagesPullingTime="2.497962263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:15:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:37.733328243 +0000 UTC m=+265.333278024" watchObservedRunningTime="2026-06-14 19:15:37.734233657 +0000 UTC m=+265.334183438" Jun 14 19:15:41.979573 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:41.979536 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:41.979959 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:41.979586 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:43.626040 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:43.626006 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:43.626410 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:43.626067 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:44.166430 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:44.166394 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:44.166620 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:44.166444 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:15:44.167903 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:44.167876 2567 patch_prober.go:28] interesting pod/console-5859b7696f-nhdpp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" start-of-body= Jun 14 19:15:44.168002 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:44.167926 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5859b7696f-nhdpp" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" probeResult="failure" output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" Jun 14 19:15:51.980085 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.980050 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:51.980421 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.980099 2567 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:51.980421 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.980136 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:15:51.980574 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.980554 2567 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"56d52b6f86b4cce9b411b22209c8090c911435b6edc27cd28ff62914a331ada4"} pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" containerMessage="Container registry failed liveness probe, will be restarted" Jun 14 19:15:51.984039 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.984013 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:15:51.984178 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:51.984063 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:15:53.374889 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:15:53.374857 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-q9m4c" podUID="915d84aa-5f57-4116-955f-f02b9743be96" Jun 14 19:15:53.773853 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:53.773774 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:15:54.167032 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:54.166999 2567 patch_prober.go:28] interesting pod/console-5859b7696f-nhdpp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" start-of-body= Jun 14 19:15:54.167180 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:54.167058 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5859b7696f-nhdpp" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" probeResult="failure" output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" Jun 14 19:15:57.179474 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.179443 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:15:57.181652 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.181621 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/915d84aa-5f57-4116-955f-f02b9743be96-metrics-tls\") pod \"dns-default-q9m4c\" (UID: \"915d84aa-5f57-4116-955f-f02b9743be96\") " pod="openshift-dns/dns-default-q9m4c" Jun 14 19:15:57.377643 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.377615 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-nf5nn\"" Jun 14 19:15:57.385082 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.385054 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:15:57.507297 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.507275 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-q9m4c"] Jun 14 19:15:57.509548 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:15:57.509523 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod915d84aa_5f57_4116_955f_f02b9743be96.slice/crio-468b9a5e663808fc3574b3030f88be8e213d8e5fe884a5d5de0bdea7e7183f33 WatchSource:0}: Error finding container 468b9a5e663808fc3574b3030f88be8e213d8e5fe884a5d5de0bdea7e7183f33: Status 404 returned error can't find the container with id 468b9a5e663808fc3574b3030f88be8e213d8e5fe884a5d5de0bdea7e7183f33 Jun 14 19:15:57.787352 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:57.787269 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9m4c" event={"ID":"915d84aa-5f57-4116-955f-f02b9743be96","Type":"ContainerStarted","Data":"468b9a5e663808fc3574b3030f88be8e213d8e5fe884a5d5de0bdea7e7183f33"} Jun 14 19:15:59.796555 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:59.796523 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9m4c" event={"ID":"915d84aa-5f57-4116-955f-f02b9743be96","Type":"ContainerStarted","Data":"ec6f79e698b8d895b6b3b1793ced0bedb3b096dade1cbf25bf6ae4a7cdde87e8"} Jun 14 19:15:59.796947 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:59.796571 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-q9m4c" event={"ID":"915d84aa-5f57-4116-955f-f02b9743be96","Type":"ContainerStarted","Data":"605ccfd189416da98aff6217ff1f504cf871a7e646c37ec2a448e0c423d3516e"} Jun 14 19:15:59.796947 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:59.796601 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:15:59.812964 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:15:59.812846 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-q9m4c" podStartSLOduration=251.363205954 podStartE2EDuration="4m12.812828674s" totalImagesPullingTime="1.44962272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:11:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:15:59.812318268 +0000 UTC m=+287.412268050" watchObservedRunningTime="2026-06-14 19:15:59.812828674 +0000 UTC m=+287.412778457" Jun 14 19:16:01.984781 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:01.984749 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:16:01.985154 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:01.984798 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:16:04.166812 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:04.166773 2567 patch_prober.go:28] interesting pod/console-5859b7696f-nhdpp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" start-of-body= Jun 14 19:16:04.167298 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:04.166825 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-5859b7696f-nhdpp" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" probeResult="failure" output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" Jun 14 19:16:09.802431 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:09.802400 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-q9m4c" Jun 14 19:16:11.984278 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:11.984243 2567 patch_prober.go:28] interesting pod/image-registry-6c6fd6c8b4-zmz6j container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 14 19:16:11.984685 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:11.984306 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 14 19:16:12.795490 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:12.795461 2567 kubelet.go:1648] "Image garbage collection succeeded" Jun 14 19:16:14.170590 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:14.170563 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:16:14.174281 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:14.174264 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:16:16.999305 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:16.999264 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" podUID="f2ad0393-407d-438f-b98b-70758d12d005" containerName="registry" containerID="cri-o://56d52b6f86b4cce9b411b22209c8090c911435b6edc27cd28ff62914a331ada4" gracePeriod=30 Jun 14 19:16:18.865858 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:18.865774 2567 generic.go:356] "Generic (PLEG): container finished" podID="f2ad0393-407d-438f-b98b-70758d12d005" containerID="56d52b6f86b4cce9b411b22209c8090c911435b6edc27cd28ff62914a331ada4" exitCode=0 Jun 14 19:16:18.866225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:18.865851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" event={"ID":"f2ad0393-407d-438f-b98b-70758d12d005","Type":"ContainerDied","Data":"56d52b6f86b4cce9b411b22209c8090c911435b6edc27cd28ff62914a331ada4"} Jun 14 19:16:18.866225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:18.865902 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" event={"ID":"f2ad0393-407d-438f-b98b-70758d12d005","Type":"ContainerStarted","Data":"4ad78635f95c89fa1b2b5f4296650370d98ce0e9b172304db98473957b5e439b"} Jun 14 19:16:18.866225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:18.866055 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:16:39.874571 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:16:39.874541 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c6fd6c8b4-zmz6j" Jun 14 19:17:08.380177 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:08.380102 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:17:33.399682 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.399628 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5859b7696f-nhdpp" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" containerID="cri-o://ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80" gracePeriod=15 Jun 14 19:17:33.643525 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.643502 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:17:33.752798 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752734 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnv5g\" (UniqueName: \"kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752798 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752788 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752806 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752823 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752852 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752884 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.752977 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.752907 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle\") pod \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\" (UID: \"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb\") " Jun 14 19:17:33.753220 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.753122 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 14 19:17:33.753334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.753306 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config" (OuterVolumeSpecName: "console-config") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 14 19:17:33.753334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.753317 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca" (OuterVolumeSpecName: "service-ca") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 14 19:17:33.753413 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.753326 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 14 19:17:33.755198 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.755172 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 14 19:17:33.755343 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.755325 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 14 19:17:33.755385 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.755360 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g" (OuterVolumeSpecName: "kube-api-access-qnv5g") pod "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" (UID: "0518e718-4ef5-49b6-8c1d-4e0dd32d35bb"). InnerVolumeSpecName "kube-api-access-qnv5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 14 19:17:33.853617 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853593 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-oauth-serving-cert\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853617 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853617 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-oauth-config\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853643 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-serving-cert\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853657 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-console-config\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853670 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-service-ca\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853680 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-trusted-ca-bundle\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:33.853780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:33.853692 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnv5g\" (UniqueName: \"kubernetes.io/projected/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb-kube-api-access-qnv5g\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:17:34.106742 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.106712 2567 generic.go:356] "Generic (PLEG): container finished" podID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerID="ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80" exitCode=0 Jun 14 19:17:34.106891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.106758 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5859b7696f-nhdpp" Jun 14 19:17:34.106891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.106790 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859b7696f-nhdpp" event={"ID":"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb","Type":"ContainerDied","Data":"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80"} Jun 14 19:17:34.106891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.106829 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5859b7696f-nhdpp" event={"ID":"0518e718-4ef5-49b6-8c1d-4e0dd32d35bb","Type":"ContainerDied","Data":"70a86a4739c6f4556fbb39311d619a105063be9757e8be250f78d8c0684dd9c3"} Jun 14 19:17:34.106891 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.106849 2567 scope.go:117] "RemoveContainer" containerID="ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80" Jun 14 19:17:34.116052 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.116035 2567 scope.go:117] "RemoveContainer" containerID="ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80" Jun 14 19:17:34.116331 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:17:34.116310 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80\": container with ID starting with ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80 not found: ID does not exist" containerID="ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80" Jun 14 19:17:34.116422 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.116336 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80"} err="failed to get container status \"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80\": rpc error: code = NotFound desc = could not find container \"ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80\": container with ID starting with ef0b77b2c839302f3c68b7f18efe1af15acedbc3f0a5bb1c82cf7eff8000fd80 not found: ID does not exist" Jun 14 19:17:34.127477 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.127455 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:17:34.131319 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.131300 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5859b7696f-nhdpp"] Jun 14 19:17:34.851430 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:17:34.851398 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" path="/var/lib/kubelet/pods/0518e718-4ef5-49b6-8c1d-4e0dd32d35bb/volumes" Jun 14 19:19:12.988844 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.988813 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4"] Jun 14 19:19:12.989246 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.989018 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" Jun 14 19:19:12.989246 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.989027 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" Jun 14 19:19:12.989246 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.989073 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="0518e718-4ef5-49b6-8c1d-4e0dd32d35bb" containerName="console" Jun 14 19:19:12.991736 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.991718 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:12.996574 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.996536 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 14 19:19:12.996815 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.996799 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 14 19:19:12.997167 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.997150 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ltfm9\"" Jun 14 19:19:12.997965 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.997942 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 14 19:19:12.998048 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:12.997975 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 14 19:19:13.022534 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.022512 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4"] Jun 14 19:19:13.092498 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.092469 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-manifests\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.092662 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.092505 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-tmp\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.092662 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.092541 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.092662 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.092611 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.092793 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.092679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhkdf\" (UniqueName: \"kubernetes.io/projected/81448164-63f1-49d6-93b3-478bdc993f61-kube-api-access-zhkdf\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.193911 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.193874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.194027 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.193923 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.194027 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.193961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhkdf\" (UniqueName: \"kubernetes.io/projected/81448164-63f1-49d6-93b3-478bdc993f61-kube-api-access-zhkdf\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.194027 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.194015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-manifests\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.194186 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.194037 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-tmp\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.194400 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.194373 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-manifests\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.196232 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.196211 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/81448164-63f1-49d6-93b3-478bdc993f61-tmp\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.196458 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.196442 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.196683 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.196662 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/81448164-63f1-49d6-93b3-478bdc993f61-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.201833 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.201814 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhkdf\" (UniqueName: \"kubernetes.io/projected/81448164-63f1-49d6-93b3-478bdc993f61-kube-api-access-zhkdf\") pod \"opendatahub-operator-controller-manager-7f8bf57587-zhrh4\" (UID: \"81448164-63f1-49d6-93b3-478bdc993f61\") " pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.300652 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.300587 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:13.426766 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.426734 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4"] Jun 14 19:19:13.430241 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:19:13.430217 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81448164_63f1_49d6_93b3_478bdc993f61.slice/crio-3ad0fc0cdb2c75e0679c3e0340918061a62cca7721df19a7e373f93b6ff06698 WatchSource:0}: Error finding container 3ad0fc0cdb2c75e0679c3e0340918061a62cca7721df19a7e373f93b6ff06698: Status 404 returned error can't find the container with id 3ad0fc0cdb2c75e0679c3e0340918061a62cca7721df19a7e373f93b6ff06698 Jun 14 19:19:13.432714 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:13.432689 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 14 19:19:14.433176 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:14.433139 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" event={"ID":"81448164-63f1-49d6-93b3-478bdc993f61","Type":"ContainerStarted","Data":"3ad0fc0cdb2c75e0679c3e0340918061a62cca7721df19a7e373f93b6ff06698"} Jun 14 19:19:16.441650 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:16.441597 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" event={"ID":"81448164-63f1-49d6-93b3-478bdc993f61","Type":"ContainerStarted","Data":"3741f79eff46016f82b45322a3957f6c9c04b3c4193b4f7b9eec8f90407d8ba4"} Jun 14 19:19:17.446081 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:17.446051 2567 generic.go:356] "Generic (PLEG): container finished" podID="81448164-63f1-49d6-93b3-478bdc993f61" containerID="3741f79eff46016f82b45322a3957f6c9c04b3c4193b4f7b9eec8f90407d8ba4" exitCode=0 Jun 14 19:19:17.446449 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:17.446094 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" event={"ID":"81448164-63f1-49d6-93b3-478bdc993f61","Type":"ContainerDied","Data":"3741f79eff46016f82b45322a3957f6c9c04b3c4193b4f7b9eec8f90407d8ba4"} Jun 14 19:19:18.452560 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:18.452528 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" event={"ID":"81448164-63f1-49d6-93b3-478bdc993f61","Type":"ContainerStarted","Data":"065803b0b5afb5759c9df90d648c787b1aa852675a9ae3404c674797fea64161"} Jun 14 19:19:18.452946 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:18.452679 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:18.474438 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:18.474378 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" podStartSLOduration=2.392363031 podStartE2EDuration="6.474357987s" totalImagesPullingTime="3.121095136s" totalInitContainerRuntime="960.89982ms" isStatefulPod=false podCreationTimestamp="2026-06-14 19:19:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:19:18.472120676 +0000 UTC m=+486.072070465" watchObservedRunningTime="2026-06-14 19:19:18.474357987 +0000 UTC m=+486.074307770" Jun 14 19:19:29.458494 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:29.458462 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7f8bf57587-zhrh4" Jun 14 19:19:31.294119 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.294086 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft"] Jun 14 19:19:31.297089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.297068 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.301090 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301035 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 14 19:19:31.301090 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301052 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 14 19:19:31.301090 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301063 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 14 19:19:31.301345 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301327 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 14 19:19:31.301407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301350 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 14 19:19:31.301407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.301396 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6cqx4\"" Jun 14 19:19:31.312474 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.312450 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft"] Jun 14 19:19:31.417907 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.417873 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.417907 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.417911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21edea9d-49c6-4449-ba43-fbeabb4afb0f-manager-config\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.418092 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.417933 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86s8t\" (UniqueName: \"kubernetes.io/projected/21edea9d-49c6-4449-ba43-fbeabb4afb0f-kube-api-access-86s8t\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.418092 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.418011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-metrics-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.518687 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.518655 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.518835 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.518692 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21edea9d-49c6-4449-ba43-fbeabb4afb0f-manager-config\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.518835 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.518716 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86s8t\" (UniqueName: \"kubernetes.io/projected/21edea9d-49c6-4449-ba43-fbeabb4afb0f-kube-api-access-86s8t\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.518835 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.518759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-metrics-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.519419 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.519393 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/21edea9d-49c6-4449-ba43-fbeabb4afb0f-manager-config\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.521079 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.521052 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-metrics-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.521166 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.521112 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/21edea9d-49c6-4449-ba43-fbeabb4afb0f-cert\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.534466 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.534444 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86s8t\" (UniqueName: \"kubernetes.io/projected/21edea9d-49c6-4449-ba43-fbeabb4afb0f-kube-api-access-86s8t\") pod \"lws-controller-manager-58b48f6fb9-6njft\" (UID: \"21edea9d-49c6-4449-ba43-fbeabb4afb0f\") " pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.606450 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.606423 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:31.747185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:31.747153 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft"] Jun 14 19:19:31.750499 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:19:31.750467 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21edea9d_49c6_4449_ba43_fbeabb4afb0f.slice/crio-621ed2a6e68ea731d631ba6673a8da80c61ce4f0e745b27c3254115cbd3fcfde WatchSource:0}: Error finding container 621ed2a6e68ea731d631ba6673a8da80c61ce4f0e745b27c3254115cbd3fcfde: Status 404 returned error can't find the container with id 621ed2a6e68ea731d631ba6673a8da80c61ce4f0e745b27c3254115cbd3fcfde Jun 14 19:19:32.500497 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:32.500427 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" event={"ID":"21edea9d-49c6-4449-ba43-fbeabb4afb0f","Type":"ContainerStarted","Data":"621ed2a6e68ea731d631ba6673a8da80c61ce4f0e745b27c3254115cbd3fcfde"} Jun 14 19:19:34.508309 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:34.508229 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" event={"ID":"21edea9d-49c6-4449-ba43-fbeabb4afb0f","Type":"ContainerStarted","Data":"438570a3276fc24bea3092bb6d3ba3c00f1bbd3e9d4c3952adad267456b659bf"} Jun 14 19:19:34.508687 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:34.508369 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:34.524881 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:34.524835 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" podStartSLOduration=1.112937996 podStartE2EDuration="3.52482156s" totalImagesPullingTime="2.411883564s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:19:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:19:34.523195338 +0000 UTC m=+502.123145118" watchObservedRunningTime="2026-06-14 19:19:34.52482156 +0000 UTC m=+502.124771554" Jun 14 19:19:45.514492 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:45.514455 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-58b48f6fb9-6njft" Jun 14 19:19:59.852321 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.852241 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7d7df895cb-s99dc"] Jun 14 19:19:59.856266 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.856251 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:19:59.859272 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.859254 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qntw8\"" Jun 14 19:19:59.860044 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.860030 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 14 19:19:59.883579 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.883557 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7d7df895cb-s99dc"] Jun 14 19:19:59.928885 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.928862 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-redirect-config\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:19:59.929005 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:19:59.928936 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnsz6\" (UniqueName: \"kubernetes.io/projected/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-kube-api-access-pnsz6\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.030147 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.030120 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-redirect-config\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.030295 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.030169 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnsz6\" (UniqueName: \"kubernetes.io/projected/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-kube-api-access-pnsz6\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.030837 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.030816 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-redirect-config\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.042320 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.042292 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnsz6\" (UniqueName: \"kubernetes.io/projected/1b5c4cfa-a53f-40b3-b5d5-4b66de14c379-kube-api-access-pnsz6\") pod \"dashboard-redirect-7d7df895cb-s99dc\" (UID: \"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379\") " pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.165160 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.165081 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" Jun 14 19:20:00.292939 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.292913 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7d7df895cb-s99dc"] Jun 14 19:20:00.295257 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:20:00.295226 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b5c4cfa_a53f_40b3_b5d5_4b66de14c379.slice/crio-fa1ef5d8baea77f0aa8cbdd577b70605819d2feab0b085efc169861855228f9b WatchSource:0}: Error finding container fa1ef5d8baea77f0aa8cbdd577b70605819d2feab0b085efc169861855228f9b: Status 404 returned error can't find the container with id fa1ef5d8baea77f0aa8cbdd577b70605819d2feab0b085efc169861855228f9b Jun 14 19:20:00.597066 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:00.597038 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" event={"ID":"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379","Type":"ContainerStarted","Data":"fa1ef5d8baea77f0aa8cbdd577b70605819d2feab0b085efc169861855228f9b"} Jun 14 19:20:05.616197 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:05.616159 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" event={"ID":"1b5c4cfa-a53f-40b3-b5d5-4b66de14c379","Type":"ContainerStarted","Data":"98ac8d00c3143fa556ad156f6fef220239f89e41bb6be7af17ee1b2aaf54aac5"} Jun 14 19:20:05.632621 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:05.632565 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7d7df895cb-s99dc" podStartSLOduration=1.848907862 podStartE2EDuration="6.632549939s" totalImagesPullingTime="4.783642077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:19:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:20:05.630971454 +0000 UTC m=+533.230921235" watchObservedRunningTime="2026-06-14 19:20:05.632549939 +0000 UTC m=+533.232499722" Jun 14 19:20:13.901046 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.901010 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv"] Jun 14 19:20:13.909550 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.909530 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:13.912108 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.912086 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-pqr24\"" Jun 14 19:20:13.912348 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.912325 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 14 19:20:13.912348 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.912342 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 14 19:20:13.912509 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.912342 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 14 19:20:13.916459 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:13.916432 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv"] Jun 14 19:20:14.035134 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035101 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035134 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035135 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035166 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035217 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035250 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035487 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035348 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035487 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035487 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035454 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq7q8\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-kube-api-access-jq7q8\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.035607 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.035486 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136344 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136311 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136355 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136415 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136433 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq7q8\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-kube-api-access-jq7q8\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136462 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136779 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136669 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136779 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136779 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136762 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136933 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136800 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136989 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136937 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.136989 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.136975 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.137158 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.137132 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-data\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.137546 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.137524 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.138952 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.138922 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.139107 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.139077 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.143754 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.143730 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-istio-token\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.144255 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.144231 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq7q8\" (UniqueName: \"kubernetes.io/projected/156d3807-3ca3-420f-aa1f-34dbd2e2f6bd-kube-api-access-jq7q8\") pod \"data-science-gateway-data-science-gateway-class-74759f9795gc4vv\" (UID: \"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.222875 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.222812 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:14.351166 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.351001 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv"] Jun 14 19:20:14.353234 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:20:14.353206 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod156d3807_3ca3_420f_aa1f_34dbd2e2f6bd.slice/crio-5f580004deb074038c9f763d92e23f8490f3b1f861d1339e9e65c50a0dd26897 WatchSource:0}: Error finding container 5f580004deb074038c9f763d92e23f8490f3b1f861d1339e9e65c50a0dd26897: Status 404 returned error can't find the container with id 5f580004deb074038c9f763d92e23f8490f3b1f861d1339e9e65c50a0dd26897 Jun 14 19:20:14.649006 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:14.648966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" event={"ID":"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd","Type":"ContainerStarted","Data":"5f580004deb074038c9f763d92e23f8490f3b1f861d1339e9e65c50a0dd26897"} Jun 14 19:20:17.432162 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:17.432117 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:20:17.432433 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:17.432193 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:20:17.432433 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:17.432223 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:20:17.664427 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:17.664384 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" event={"ID":"156d3807-3ca3-420f-aa1f-34dbd2e2f6bd","Type":"ContainerStarted","Data":"67f666bb2f8a25be0c81672ffbd061690ebd80e3d70a7cf293ece484a4426e5d"} Jun 14 19:20:17.691500 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:17.691241 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" podStartSLOduration=1.61468567 podStartE2EDuration="4.691221772s" totalImagesPullingTime="3.076536102s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:20:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:20:17.690627234 +0000 UTC m=+545.290577017" watchObservedRunningTime="2026-06-14 19:20:17.691221772 +0000 UTC m=+545.291171555" Jun 14 19:20:18.223388 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:18.223359 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:18.227873 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:18.227851 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:18.668963 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:18.668927 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:18.669758 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:18.669740 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-74759f9795gc4vv" Jun 14 19:20:33.014089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.014057 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:33.017046 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.017012 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:33.019748 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.019729 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 14 19:20:33.019863 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.019764 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 14 19:20:33.019930 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.019802 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-kjts4\"" Jun 14 19:20:33.027047 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.027029 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:33.179490 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.179458 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsl89\" (UniqueName: \"kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89\") pod \"kuadrant-operator-catalog-cjc7r\" (UID: \"f402ae56-79b7-4743-85af-9ac796136906\") " pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:33.280287 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.280211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsl89\" (UniqueName: \"kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89\") pod \"kuadrant-operator-catalog-cjc7r\" (UID: \"f402ae56-79b7-4743-85af-9ac796136906\") " pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:33.289371 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.289349 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsl89\" (UniqueName: \"kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89\") pod \"kuadrant-operator-catalog-cjc7r\" (UID: \"f402ae56-79b7-4743-85af-9ac796136906\") " pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:33.328896 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.328875 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:33.384129 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.384101 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:33.450193 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.450159 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:33.597778 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.597747 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2xsps"] Jun 14 19:20:33.601894 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.601876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:33.611293 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.611262 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2xsps"] Jun 14 19:20:33.722483 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.722454 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" event={"ID":"f402ae56-79b7-4743-85af-9ac796136906","Type":"ContainerStarted","Data":"6c904ca951e3663bc850d625f9d16a1861ab3c01e0937fcac6792b0e804dd114"} Jun 14 19:20:33.785508 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.785478 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbrq2\" (UniqueName: \"kubernetes.io/projected/2491c3cf-5113-41df-987c-1eab9242dfb4-kube-api-access-vbrq2\") pod \"kuadrant-operator-catalog-2xsps\" (UID: \"2491c3cf-5113-41df-987c-1eab9242dfb4\") " pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:33.886721 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.886630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbrq2\" (UniqueName: \"kubernetes.io/projected/2491c3cf-5113-41df-987c-1eab9242dfb4-kube-api-access-vbrq2\") pod \"kuadrant-operator-catalog-2xsps\" (UID: \"2491c3cf-5113-41df-987c-1eab9242dfb4\") " pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:33.896406 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.896378 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbrq2\" (UniqueName: \"kubernetes.io/projected/2491c3cf-5113-41df-987c-1eab9242dfb4-kube-api-access-vbrq2\") pod \"kuadrant-operator-catalog-2xsps\" (UID: \"2491c3cf-5113-41df-987c-1eab9242dfb4\") " pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:33.911105 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:33.911074 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:34.055266 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:34.055244 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-2xsps"] Jun 14 19:20:34.067325 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:20:34.067300 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2491c3cf_5113_41df_987c_1eab9242dfb4.slice/crio-673c9904b15f4b0eeea3bbaa4ab88f57cad2b7ddd397ec7b912a4d86b056f64c WatchSource:0}: Error finding container 673c9904b15f4b0eeea3bbaa4ab88f57cad2b7ddd397ec7b912a4d86b056f64c: Status 404 returned error can't find the container with id 673c9904b15f4b0eeea3bbaa4ab88f57cad2b7ddd397ec7b912a4d86b056f64c Jun 14 19:20:34.729997 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:34.729949 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" event={"ID":"2491c3cf-5113-41df-987c-1eab9242dfb4","Type":"ContainerStarted","Data":"673c9904b15f4b0eeea3bbaa4ab88f57cad2b7ddd397ec7b912a4d86b056f64c"} Jun 14 19:20:36.747721 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.747681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" event={"ID":"f402ae56-79b7-4743-85af-9ac796136906","Type":"ContainerStarted","Data":"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370"} Jun 14 19:20:36.748164 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.747819 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" podUID="f402ae56-79b7-4743-85af-9ac796136906" containerName="registry-server" containerID="cri-o://37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370" gracePeriod=2 Jun 14 19:20:36.749341 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.749312 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" event={"ID":"2491c3cf-5113-41df-987c-1eab9242dfb4","Type":"ContainerStarted","Data":"f728ef2d3dcd0dd2f80c0c58502bf45f78de44e762e8de1e88b8ca731d59e7f1"} Jun 14 19:20:36.768620 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.768583 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" podStartSLOduration=2.55888314 podStartE2EDuration="4.768570408s" totalImagesPullingTime="2.209687268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:20:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:20:36.766796958 +0000 UTC m=+564.366746738" watchObservedRunningTime="2026-06-14 19:20:36.768570408 +0000 UTC m=+564.368520188" Jun 14 19:20:36.786901 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.786863 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" podStartSLOduration=2.187633171 podStartE2EDuration="3.786850964s" totalImagesPullingTime="1.599217793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:20:36.786587018 +0000 UTC m=+564.386536817" watchObservedRunningTime="2026-06-14 19:20:36.786850964 +0000 UTC m=+564.386800744" Jun 14 19:20:36.982346 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:36.982325 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:37.007747 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.007694 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsl89\" (UniqueName: \"kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89\") pod \"f402ae56-79b7-4743-85af-9ac796136906\" (UID: \"f402ae56-79b7-4743-85af-9ac796136906\") " Jun 14 19:20:37.009792 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.009766 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89" (OuterVolumeSpecName: "kube-api-access-lsl89") pod "f402ae56-79b7-4743-85af-9ac796136906" (UID: "f402ae56-79b7-4743-85af-9ac796136906"). InnerVolumeSpecName "kube-api-access-lsl89". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 14 19:20:37.108401 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.108376 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsl89\" (UniqueName: \"kubernetes.io/projected/f402ae56-79b7-4743-85af-9ac796136906-kube-api-access-lsl89\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:20:37.754757 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.754721 2567 generic.go:356] "Generic (PLEG): container finished" podID="f402ae56-79b7-4743-85af-9ac796136906" containerID="37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370" exitCode=0 Jun 14 19:20:37.755156 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.754764 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" Jun 14 19:20:37.755156 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.754804 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" event={"ID":"f402ae56-79b7-4743-85af-9ac796136906","Type":"ContainerDied","Data":"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370"} Jun 14 19:20:37.755156 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.754853 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-cjc7r" event={"ID":"f402ae56-79b7-4743-85af-9ac796136906","Type":"ContainerDied","Data":"6c904ca951e3663bc850d625f9d16a1861ab3c01e0937fcac6792b0e804dd114"} Jun 14 19:20:37.755156 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.754881 2567 scope.go:117] "RemoveContainer" containerID="37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370" Jun 14 19:20:37.763836 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.763821 2567 scope.go:117] "RemoveContainer" containerID="37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370" Jun 14 19:20:37.764070 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:20:37.764054 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370\": container with ID starting with 37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370 not found: ID does not exist" containerID="37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370" Jun 14 19:20:37.764134 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.764077 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370"} err="failed to get container status \"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370\": rpc error: code = NotFound desc = could not find container \"37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370\": container with ID starting with 37782a24ab0c60936f1db3a4dcb6549404be4042da9512457a03f3b82ce11370 not found: ID does not exist" Jun 14 19:20:37.776122 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.776102 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:37.781396 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:37.781372 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-cjc7r"] Jun 14 19:20:38.851597 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:38.851567 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f402ae56-79b7-4743-85af-9ac796136906" path="/var/lib/kubelet/pods/f402ae56-79b7-4743-85af-9ac796136906/volumes" Jun 14 19:20:43.911914 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:43.911878 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:43.912330 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:43.911940 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:43.933612 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:43.933587 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:20:44.803971 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:20:44.803945 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-2xsps" Jun 14 19:21:00.040391 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.040361 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll"] Jun 14 19:21:00.040853 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.040686 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f402ae56-79b7-4743-85af-9ac796136906" containerName="registry-server" Jun 14 19:21:00.040853 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.040700 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="f402ae56-79b7-4743-85af-9ac796136906" containerName="registry-server" Jun 14 19:21:00.040853 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.040750 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="f402ae56-79b7-4743-85af-9ac796136906" containerName="registry-server" Jun 14 19:21:00.054565 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.054541 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:00.057209 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.057186 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll"] Jun 14 19:21:00.057828 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.057810 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 14 19:21:00.058245 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.058225 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qsn5q\"" Jun 14 19:21:00.147195 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.147170 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfhnh\" (UniqueName: \"kubernetes.io/projected/3a07c031-874d-40c5-8c1e-6ea42197a114-kube-api-access-qfhnh\") pod \"dns-operator-controller-manager-864dd78c4c-5vpll\" (UID: \"3a07c031-874d-40c5-8c1e-6ea42197a114\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:00.248234 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.248211 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfhnh\" (UniqueName: \"kubernetes.io/projected/3a07c031-874d-40c5-8c1e-6ea42197a114-kube-api-access-qfhnh\") pod \"dns-operator-controller-manager-864dd78c4c-5vpll\" (UID: \"3a07c031-874d-40c5-8c1e-6ea42197a114\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:00.258999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.258973 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfhnh\" (UniqueName: \"kubernetes.io/projected/3a07c031-874d-40c5-8c1e-6ea42197a114-kube-api-access-qfhnh\") pod \"dns-operator-controller-manager-864dd78c4c-5vpll\" (UID: \"3a07c031-874d-40c5-8c1e-6ea42197a114\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:00.365132 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.365097 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:00.505018 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.504990 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll"] Jun 14 19:21:00.508157 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:00.508129 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a07c031_874d_40c5_8c1e_6ea42197a114.slice/crio-362fa97349938b44a10a0579c08989403e1d58c8dca14708d9a41b1311df6f3f WatchSource:0}: Error finding container 362fa97349938b44a10a0579c08989403e1d58c8dca14708d9a41b1311df6f3f: Status 404 returned error can't find the container with id 362fa97349938b44a10a0579c08989403e1d58c8dca14708d9a41b1311df6f3f Jun 14 19:21:00.840937 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:00.840905 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" event={"ID":"3a07c031-874d-40c5-8c1e-6ea42197a114","Type":"ContainerStarted","Data":"362fa97349938b44a10a0579c08989403e1d58c8dca14708d9a41b1311df6f3f"} Jun 14 19:21:03.702402 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.702370 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9"] Jun 14 19:21:03.705440 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.705423 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:03.708242 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.708224 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-hp65h\"" Jun 14 19:21:03.716821 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.716801 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9"] Jun 14 19:21:03.775328 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.775302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fs7bm\" (UniqueName: \"kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm\") pod \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" (UID: \"c026bf7f-e928-4b38-9fa4-6da731b2f674\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:03.854581 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.854550 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" event={"ID":"3a07c031-874d-40c5-8c1e-6ea42197a114","Type":"ContainerStarted","Data":"2e2d03b729a50b8c819cd4b97002f964dc2c58dc60e7aa561c0f0dadfced3183"} Jun 14 19:21:03.854710 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.854693 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:03.876090 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.876064 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fs7bm\" (UniqueName: \"kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm\") pod \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" (UID: \"c026bf7f-e928-4b38-9fa4-6da731b2f674\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:03.877954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.877917 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" podStartSLOduration=1.434325092 podStartE2EDuration="3.877904674s" totalImagesPullingTime="2.443579582s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:03.876469433 +0000 UTC m=+591.476419231" watchObservedRunningTime="2026-06-14 19:21:03.877904674 +0000 UTC m=+591.477854511" Jun 14 19:21:03.889640 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:03.889616 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fs7bm\" (UniqueName: \"kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm\") pod \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" (UID: \"c026bf7f-e928-4b38-9fa4-6da731b2f674\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:04.014563 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:04.014495 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:04.141868 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:04.141836 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9"] Jun 14 19:21:04.145266 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:04.145234 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc026bf7f_e928_4b38_9fa4_6da731b2f674.slice/crio-664b67beb6d18834b240553877e5a30e6329db314aec690bcecc3197ca49d6e3 WatchSource:0}: Error finding container 664b67beb6d18834b240553877e5a30e6329db314aec690bcecc3197ca49d6e3: Status 404 returned error can't find the container with id 664b67beb6d18834b240553877e5a30e6329db314aec690bcecc3197ca49d6e3 Jun 14 19:21:04.861502 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:04.861455 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" event={"ID":"c026bf7f-e928-4b38-9fa4-6da731b2f674","Type":"ContainerStarted","Data":"664b67beb6d18834b240553877e5a30e6329db314aec690bcecc3197ca49d6e3"} Jun 14 19:21:06.872693 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:06.872660 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" event={"ID":"c026bf7f-e928-4b38-9fa4-6da731b2f674","Type":"ContainerStarted","Data":"051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527"} Jun 14 19:21:06.873136 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:06.872769 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:06.897908 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:06.897853 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" podStartSLOduration=1.9286477359999998 podStartE2EDuration="3.89784015s" totalImagesPullingTime="1.969192414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:06.896703115 +0000 UTC m=+594.496652918" watchObservedRunningTime="2026-06-14 19:21:06.89784015 +0000 UTC m=+594.497789923" Jun 14 19:21:09.133195 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.133159 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9"] Jun 14 19:21:09.136067 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.136050 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.139275 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.139252 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-dvfvh\"" Jun 14 19:21:09.158432 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.158410 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9"] Jun 14 19:21:09.210609 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.210578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htlrb\" (UniqueName: \"kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.210792 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.210617 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.311407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.311358 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htlrb\" (UniqueName: \"kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.311407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.311408 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.311770 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.311754 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.325267 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.325237 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-htlrb\" (UniqueName: \"kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb\") pod \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.446030 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.445965 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:09.591455 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.591430 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9"] Jun 14 19:21:09.594449 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:09.594424 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b19b531_6fb4_4898_820c_171348f3c336.slice/crio-52084c368b58409d16eb8faad47da125a2087beabbff1d6f9ec5508e8501d784 WatchSource:0}: Error finding container 52084c368b58409d16eb8faad47da125a2087beabbff1d6f9ec5508e8501d784: Status 404 returned error can't find the container with id 52084c368b58409d16eb8faad47da125a2087beabbff1d6f9ec5508e8501d784 Jun 14 19:21:09.885740 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:09.885712 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" event={"ID":"5b19b531-6fb4-4898-820c-171348f3c336","Type":"ContainerStarted","Data":"52084c368b58409d16eb8faad47da125a2087beabbff1d6f9ec5508e8501d784"} Jun 14 19:21:14.864830 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:14.864786 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-5vpll" Jun 14 19:21:17.879625 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:17.879597 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:17.924317 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:17.924276 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" event={"ID":"5b19b531-6fb4-4898-820c-171348f3c336","Type":"ContainerStarted","Data":"3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980"} Jun 14 19:21:17.924530 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:17.924511 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:17.951345 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:17.951292 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" podStartSLOduration=1.5298407809999999 podStartE2EDuration="8.951273765s" totalImagesPullingTime="7.421432984s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:17.950618056 +0000 UTC m=+605.550567856" watchObservedRunningTime="2026-06-14 19:21:17.951273765 +0000 UTC m=+605.551223532" Jun 14 19:21:19.606437 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.606403 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-584xb"] Jun 14 19:21:19.609619 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.609603 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.612321 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.612300 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 14 19:21:19.612452 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.612345 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-s865v\"" Jun 14 19:21:19.612452 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.612345 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 14 19:21:19.617443 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.617421 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-584xb"] Jun 14 19:21:19.691624 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.691594 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f44d420-3920-476a-b44f-806e7c388e21-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.691773 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.691653 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.691773 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.691709 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jwkn\" (UniqueName: \"kubernetes.io/projected/2f44d420-3920-476a-b44f-806e7c388e21-kube-api-access-8jwkn\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.792169 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.792135 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jwkn\" (UniqueName: \"kubernetes.io/projected/2f44d420-3920-476a-b44f-806e7c388e21-kube-api-access-8jwkn\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.792318 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.792201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f44d420-3920-476a-b44f-806e7c388e21-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.792318 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.792234 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.792318 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:19.792310 2567 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jun 14 19:21:19.792436 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:19.792361 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert podName:2f44d420-3920-476a-b44f-806e7c388e21 nodeName:}" failed. No retries permitted until 2026-06-14 19:21:20.292344457 +0000 UTC m=+607.892294216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-584xb" (UID: "2f44d420-3920-476a-b44f-806e7c388e21") : secret "plugin-serving-cert" not found Jun 14 19:21:19.792893 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.792873 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/2f44d420-3920-476a-b44f-806e7c388e21-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:19.801425 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:19.801400 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jwkn\" (UniqueName: \"kubernetes.io/projected/2f44d420-3920-476a-b44f-806e7c388e21-kube-api-access-8jwkn\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:20.296334 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:20.296299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:20.298693 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:20.298674 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/2f44d420-3920-476a-b44f-806e7c388e21-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-584xb\" (UID: \"2f44d420-3920-476a-b44f-806e7c388e21\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:20.519522 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:20.519491 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" Jun 14 19:21:20.646935 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:20.646902 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-584xb"] Jun 14 19:21:20.649865 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:20.649839 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2f44d420_3920_476a_b44f_806e7c388e21.slice/crio-ca2da14f7cc3f049edf14c97f8f673e2fe6d15ea9c7e12e5210a8aa6716d39a7 WatchSource:0}: Error finding container ca2da14f7cc3f049edf14c97f8f673e2fe6d15ea9c7e12e5210a8aa6716d39a7: Status 404 returned error can't find the container with id ca2da14f7cc3f049edf14c97f8f673e2fe6d15ea9c7e12e5210a8aa6716d39a7 Jun 14 19:21:20.936117 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:20.936037 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" event={"ID":"2f44d420-3920-476a-b44f-806e7c388e21","Type":"ContainerStarted","Data":"ca2da14f7cc3f049edf14c97f8f673e2fe6d15ea9c7e12e5210a8aa6716d39a7"} Jun 14 19:21:28.931059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:28.931023 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:30.620810 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.620774 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9"] Jun 14 19:21:30.621288 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.621001 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" podUID="5b19b531-6fb4-4898-820c-171348f3c336" containerName="manager" containerID="cri-o://3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980" gracePeriod=2 Jun 14 19:21:30.623738 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.623688 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9"] Jun 14 19:21:30.630898 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.630786 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:30.647054 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.647026 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9"] Jun 14 19:21:30.647289 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.647260 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" containerName="manager" containerID="cri-o://051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527" gracePeriod=2 Jun 14 19:21:30.674185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.674105 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9"] Jun 14 19:21:30.676201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.675677 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:30.676201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676096 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" containerName="manager" Jun 14 19:21:30.676201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676113 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" containerName="manager" Jun 14 19:21:30.676201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676127 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b19b531-6fb4-4898-820c-171348f3c336" containerName="manager" Jun 14 19:21:30.676201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676138 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b19b531-6fb4-4898-820c-171348f3c336" containerName="manager" Jun 14 19:21:30.676456 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676209 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" containerName="manager" Jun 14 19:21:30.676456 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.676220 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b19b531-6fb4-4898-820c-171348f3c336" containerName="manager" Jun 14 19:21:30.678092 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.678044 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"c026bf7f-e928-4b38-9fa4-6da731b2f674\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"limitador-operator-controller-manager-859d7b78bd-pq5z9\": pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" not found" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:30.679353 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.679332 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.680722 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.680689 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:30.702753 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.702729 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l"] Jun 14 19:21:30.706274 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.706253 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:30.706381 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.706357 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:30.721148 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.721123 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l"] Jun 14 19:21:30.728865 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.728832 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:30.731434 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.731401 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:30.780987 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.780955 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:30.783620 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:30.783568 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:30.787947 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.787923 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.788035 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.787964 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d276w\" (UniqueName: \"kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.788035 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.787990 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjxmq\" (UniqueName: \"kubernetes.io/projected/877bb89a-6022-4c8f-89e2-cc07d12009e2-kube-api-access-fjxmq\") pod \"limitador-operator-controller-manager-859d7b78bd-97n5l\" (UID: \"877bb89a-6022-4c8f-89e2-cc07d12009e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:30.888926 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.888696 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.888926 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.888764 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d276w\" (UniqueName: \"kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.888926 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.888811 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjxmq\" (UniqueName: \"kubernetes.io/projected/877bb89a-6022-4c8f-89e2-cc07d12009e2-kube-api-access-fjxmq\") pod \"limitador-operator-controller-manager-859d7b78bd-97n5l\" (UID: \"877bb89a-6022-4c8f-89e2-cc07d12009e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:30.889297 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.889271 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:30.902436 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.902409 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjxmq\" (UniqueName: \"kubernetes.io/projected/877bb89a-6022-4c8f-89e2-cc07d12009e2-kube-api-access-fjxmq\") pod \"limitador-operator-controller-manager-859d7b78bd-97n5l\" (UID: \"877bb89a-6022-4c8f-89e2-cc07d12009e2\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:30.902805 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:30.902776 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d276w\" (UniqueName: \"kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m78mb\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:31.041100 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:31.041060 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:31.049558 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:31.049527 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:42.309010 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.308775 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l"] Jun 14 19:21:42.311690 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:42.311663 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod877bb89a_6022_4c8f_89e2_cc07d12009e2.slice/crio-4cf2af41c46dc75acaaae80b1ee93ae3b4cabdf3b43dfeee8df5977541226658 WatchSource:0}: Error finding container 4cf2af41c46dc75acaaae80b1ee93ae3b4cabdf3b43dfeee8df5977541226658: Status 404 returned error can't find the container with id 4cf2af41c46dc75acaaae80b1ee93ae3b4cabdf3b43dfeee8df5977541226658 Jun 14 19:21:42.316108 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.316051 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:42.318657 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:42.318603 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:42.319372 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.319353 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:42.321436 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:42.321411 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:42.323651 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:42.323613 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:42.339077 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.339050 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:42.350775 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:21:42.350754 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbedf4c72_3f18_4b52_bccf_7bb5610d6494.slice/crio-23ae12aba86972b5daa1b467e85cf824e6a442c568b5d52a3c7664ccbf960f6a WatchSource:0}: Error finding container 23ae12aba86972b5daa1b467e85cf824e6a442c568b5d52a3c7664ccbf960f6a: Status 404 returned error can't find the container with id 23ae12aba86972b5daa1b467e85cf824e6a442c568b5d52a3c7664ccbf960f6a Jun 14 19:21:42.391686 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.391661 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htlrb\" (UniqueName: \"kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb\") pod \"5b19b531-6fb4-4898-820c-171348f3c336\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " Jun 14 19:21:42.391798 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.391714 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume\") pod \"5b19b531-6fb4-4898-820c-171348f3c336\" (UID: \"5b19b531-6fb4-4898-820c-171348f3c336\") " Jun 14 19:21:42.391798 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.391753 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fs7bm\" (UniqueName: \"kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm\") pod \"c026bf7f-e928-4b38-9fa4-6da731b2f674\" (UID: \"c026bf7f-e928-4b38-9fa4-6da731b2f674\") " Jun 14 19:21:42.392199 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.392177 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5b19b531-6fb4-4898-820c-171348f3c336" (UID: "5b19b531-6fb4-4898-820c-171348f3c336"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 14 19:21:42.393571 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.393525 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb" (OuterVolumeSpecName: "kube-api-access-htlrb") pod "5b19b531-6fb4-4898-820c-171348f3c336" (UID: "5b19b531-6fb4-4898-820c-171348f3c336"). InnerVolumeSpecName "kube-api-access-htlrb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 14 19:21:42.393670 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.393581 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm" (OuterVolumeSpecName: "kube-api-access-fs7bm") pod "c026bf7f-e928-4b38-9fa4-6da731b2f674" (UID: "c026bf7f-e928-4b38-9fa4-6da731b2f674"). InnerVolumeSpecName "kube-api-access-fs7bm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 14 19:21:42.493157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.493113 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htlrb\" (UniqueName: \"kubernetes.io/projected/5b19b531-6fb4-4898-820c-171348f3c336-kube-api-access-htlrb\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:21:42.493157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.493153 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5b19b531-6fb4-4898-820c-171348f3c336-extensions-socket-volume\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:21:42.493157 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.493162 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fs7bm\" (UniqueName: \"kubernetes.io/projected/c026bf7f-e928-4b38-9fa4-6da731b2f674-kube-api-access-fs7bm\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:21:42.854668 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:42.854614 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:42.856869 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:42.856838 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:42.857672 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.857650 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b19b531-6fb4-4898-820c-171348f3c336" path="/var/lib/kubelet/pods/5b19b531-6fb4-4898-820c-171348f3c336/volumes" Jun 14 19:21:42.857963 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:42.857949 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" path="/var/lib/kubelet/pods/c026bf7f-e928-4b38-9fa4-6da731b2f674/volumes" Jun 14 19:21:43.046503 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.046475 2567 generic.go:356] "Generic (PLEG): container finished" podID="c026bf7f-e928-4b38-9fa4-6da731b2f674" containerID="051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527" exitCode=0 Jun 14 19:21:43.046668 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.046533 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:43.046668 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.046554 2567 scope.go:117] "RemoveContainer" containerID="051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527" Jun 14 19:21:43.048080 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.048056 2567 generic.go:356] "Generic (PLEG): container finished" podID="5b19b531-6fb4-4898-820c-171348f3c336" containerID="3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980" exitCode=0 Jun 14 19:21:43.048193 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.048111 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:43.049166 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.049135 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:43.050155 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.050110 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" event={"ID":"bedf4c72-3f18-4b52-bccf-7bb5610d6494","Type":"ContainerStarted","Data":"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f"} Jun 14 19:21:43.050251 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.050165 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" event={"ID":"bedf4c72-3f18-4b52-bccf-7bb5610d6494","Type":"ContainerStarted","Data":"23ae12aba86972b5daa1b467e85cf824e6a442c568b5d52a3c7664ccbf960f6a"} Jun 14 19:21:43.050313 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.050273 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:43.051854 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.051824 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:43.051971 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.051949 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" event={"ID":"877bb89a-6022-4c8f-89e2-cc07d12009e2","Type":"ContainerStarted","Data":"c17b022c76699bc766f627dfbbdbf5a1eec3d54a1fbc1ed4164665c4bfe1ebfd"} Jun 14 19:21:43.052041 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.051989 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" event={"ID":"877bb89a-6022-4c8f-89e2-cc07d12009e2","Type":"ContainerStarted","Data":"4cf2af41c46dc75acaaae80b1ee93ae3b4cabdf3b43dfeee8df5977541226658"} Jun 14 19:21:43.052041 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.052028 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:21:43.053436 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.053408 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" event={"ID":"2f44d420-3920-476a-b44f-806e7c388e21","Type":"ContainerStarted","Data":"d835a7fd1b13875fce508a7841a57929c429d2d6579002621a12536145ece2e2"} Jun 14 19:21:43.055069 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.055043 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:43.055536 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.055522 2567 scope.go:117] "RemoveContainer" containerID="051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527" Jun 14 19:21:43.055795 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.055775 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527\": container with ID starting with 051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527 not found: ID does not exist" containerID="051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527" Jun 14 19:21:43.055857 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.055803 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527"} err="failed to get container status \"051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527\": rpc error: code = NotFound desc = could not find container \"051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527\": container with ID starting with 051f2880deaee1d2db4677fc2310a5951416669dcb4d014ac9309d7da3d4c527 not found: ID does not exist" Jun 14 19:21:43.055857 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.055821 2567 scope.go:117] "RemoveContainer" containerID="3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980" Jun 14 19:21:43.057388 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.057370 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-pq5z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="c026bf7f-e928-4b38-9fa4-6da731b2f674" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-pq5z9" Jun 14 19:21:43.059454 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.059432 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-mv6z9\" is forbidden: User \"system:node:ip-10-0-135-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-135-103.ec2.internal' and this object" podUID="5b19b531-6fb4-4898-820c-171348f3c336" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-mv6z9" Jun 14 19:21:43.063225 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.063211 2567 scope.go:117] "RemoveContainer" containerID="3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980" Jun 14 19:21:43.063452 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:43.063436 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980\": container with ID starting with 3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980 not found: ID does not exist" containerID="3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980" Jun 14 19:21:43.063494 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.063457 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980"} err="failed to get container status \"3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980\": rpc error: code = NotFound desc = could not find container \"3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980\": container with ID starting with 3b84736e35b0cc1034d98f702ca20388f9a686ffe16fd1f6b6cf6c2587790980 not found: ID does not exist" Jun 14 19:21:43.077942 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.077904 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-584xb" podStartSLOduration=2.517765712 podStartE2EDuration="24.077893449s" totalImagesPullingTime="21.560127737s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:43.077433174 +0000 UTC m=+630.677382974" watchObservedRunningTime="2026-06-14 19:21:43.077893449 +0000 UTC m=+630.677843230" Jun 14 19:21:43.097254 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.097215 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" podStartSLOduration=13.097202504 podStartE2EDuration="13.097202504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:43.096183851 +0000 UTC m=+630.696133632" watchObservedRunningTime="2026-06-14 19:21:43.097202504 +0000 UTC m=+630.697152284" Jun 14 19:21:43.118353 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:43.118287 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" podStartSLOduration=13.118274648 podStartE2EDuration="13.118274648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:21:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:21:43.117979293 +0000 UTC m=+630.717929103" watchObservedRunningTime="2026-06-14 19:21:43.118274648 +0000 UTC m=+630.718224428" Jun 14 19:21:46.742467 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:46.742435 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:46.742867 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:46.742658 2567 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" podUID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" containerName="manager" containerID="cri-o://7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f" gracePeriod=10 Jun 14 19:21:46.978188 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:46.978164 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:47.072590 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.072557 2567 generic.go:356] "Generic (PLEG): container finished" podID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" containerID="7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f" exitCode=0 Jun 14 19:21:47.072756 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.072601 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" Jun 14 19:21:47.072756 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.072658 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" event={"ID":"bedf4c72-3f18-4b52-bccf-7bb5610d6494","Type":"ContainerDied","Data":"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f"} Jun 14 19:21:47.072756 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.072710 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb" event={"ID":"bedf4c72-3f18-4b52-bccf-7bb5610d6494","Type":"ContainerDied","Data":"23ae12aba86972b5daa1b467e85cf824e6a442c568b5d52a3c7664ccbf960f6a"} Jun 14 19:21:47.072756 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.072733 2567 scope.go:117] "RemoveContainer" containerID="7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f" Jun 14 19:21:47.080357 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.080336 2567 scope.go:117] "RemoveContainer" containerID="7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f" Jun 14 19:21:47.080612 ip-10-0-135-103 kubenswrapper[2567]: E0614 19:21:47.080595 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f\": container with ID starting with 7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f not found: ID does not exist" containerID="7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f" Jun 14 19:21:47.080695 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.080624 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f"} err="failed to get container status \"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f\": rpc error: code = NotFound desc = could not find container \"7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f\": container with ID starting with 7c8d2b44296ffa453412a3f44de61bc2475c1c221aba869f9ae2e836660c5e5f not found: ID does not exist" Jun 14 19:21:47.127951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.127929 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume\") pod \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " Jun 14 19:21:47.128036 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.128012 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d276w\" (UniqueName: \"kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w\") pod \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\" (UID: \"bedf4c72-3f18-4b52-bccf-7bb5610d6494\") " Jun 14 19:21:47.128259 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.128237 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "bedf4c72-3f18-4b52-bccf-7bb5610d6494" (UID: "bedf4c72-3f18-4b52-bccf-7bb5610d6494"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 14 19:21:47.129899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.129878 2567 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w" (OuterVolumeSpecName: "kube-api-access-d276w") pod "bedf4c72-3f18-4b52-bccf-7bb5610d6494" (UID: "bedf4c72-3f18-4b52-bccf-7bb5610d6494"). InnerVolumeSpecName "kube-api-access-d276w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 14 19:21:47.229402 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.229378 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bedf4c72-3f18-4b52-bccf-7bb5610d6494-extensions-socket-volume\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:21:47.229402 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.229401 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d276w\" (UniqueName: \"kubernetes.io/projected/bedf4c72-3f18-4b52-bccf-7bb5610d6494-kube-api-access-d276w\") on node \"ip-10-0-135-103.ec2.internal\" DevicePath \"\"" Jun 14 19:21:47.396099 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.396072 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:47.402484 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:47.402466 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m78mb"] Jun 14 19:21:48.852753 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:48.852722 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" path="/var/lib/kubelet/pods/bedf4c72-3f18-4b52-bccf-7bb5610d6494/volumes" Jun 14 19:21:54.061880 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:21:54.061851 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-97n5l" Jun 14 19:22:03.105111 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.105074 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2"] Jun 14 19:22:03.105475 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.105360 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" containerName="manager" Jun 14 19:22:03.105475 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.105393 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" containerName="manager" Jun 14 19:22:03.105475 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.105459 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="bedf4c72-3f18-4b52-bccf-7bb5610d6494" containerName="manager" Jun 14 19:22:03.130092 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.130051 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2"] Jun 14 19:22:03.130267 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.130158 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.132817 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.132795 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-8dzpj\"" Jun 14 19:22:03.251924 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.251892 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339c850a-81ed-487a-a976-f13b3324821c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252083 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.251932 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339c850a-81ed-487a-a976-f13b3324821c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252083 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.251961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252083 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252021 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252083 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252079 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252215 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252112 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252215 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252136 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qgbt\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-kube-api-access-5qgbt\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252215 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252158 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.252215 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.252173 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.352794 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.352755 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.352975 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.352809 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qgbt\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-kube-api-access-5qgbt\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.352975 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.352838 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.352975 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.352865 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.352975 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.352891 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339c850a-81ed-487a-a976-f13b3324821c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353011 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339c850a-81ed-487a-a976-f13b3324821c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353060 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353088 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353131 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353216 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353254 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353407 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353393 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353559 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353510 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.353717 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.353694 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/339c850a-81ed-487a-a976-f13b3324821c-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.355314 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.355266 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/339c850a-81ed-487a-a976-f13b3324821c-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.355391 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.355320 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/339c850a-81ed-487a-a976-f13b3324821c-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.363899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.363879 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qgbt\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-kube-api-access-5qgbt\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.365155 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.365138 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/339c850a-81ed-487a-a976-f13b3324821c-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-v6kg2\" (UID: \"339c850a-81ed-487a-a976-f13b3324821c\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.440863 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.440826 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:03.606840 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.606626 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2"] Jun 14 19:22:03.609621 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:22:03.609587 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod339c850a_81ed_487a_a976_f13b3324821c.slice/crio-0cf858aff65a20a22bff113381aef41746a5da00427cfbd10f2a2ec2aaec03ba WatchSource:0}: Error finding container 0cf858aff65a20a22bff113381aef41746a5da00427cfbd10f2a2ec2aaec03ba: Status 404 returned error can't find the container with id 0cf858aff65a20a22bff113381aef41746a5da00427cfbd10f2a2ec2aaec03ba Jun 14 19:22:03.611804 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.611770 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:22:03.611951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.611836 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:22:03.611951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:03.611866 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:22:04.146162 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:04.146121 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" event={"ID":"339c850a-81ed-487a-a976-f13b3324821c","Type":"ContainerStarted","Data":"9ea4fda1f39e7ce1cda751106b8ccad63088ca0b882e502480caff11eb951e61"} Jun 14 19:22:04.146539 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:04.146175 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" event={"ID":"339c850a-81ed-487a-a976-f13b3324821c","Type":"ContainerStarted","Data":"0cf858aff65a20a22bff113381aef41746a5da00427cfbd10f2a2ec2aaec03ba"} Jun 14 19:22:04.169298 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:04.169250 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" podStartSLOduration=1.169234334 podStartE2EDuration="1.169234334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:22:04.167081413 +0000 UTC m=+651.767031195" watchObservedRunningTime="2026-06-14 19:22:04.169234334 +0000 UTC m=+651.769184114" Jun 14 19:22:04.441619 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:04.441534 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:04.446380 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:04.446359 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:05.150425 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:05.150391 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:05.151376 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:05.151358 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:22:06.257877 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.257844 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f47cc9f65-8474r"] Jun 14 19:22:06.337884 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.337855 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f47cc9f65-8474r"] Jun 14 19:22:06.338033 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.337939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.340932 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.340878 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 14 19:22:06.340932 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.340892 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 14 19:22:06.340932 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.340922 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 14 19:22:06.341171 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.341000 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 14 19:22:06.341171 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.341052 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 14 19:22:06.342099 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.342075 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mxpzn\"" Jun 14 19:22:06.342219 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.342129 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 14 19:22:06.342300 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.342283 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 14 19:22:06.346282 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.346264 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 14 19:22:06.375671 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375627 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-console-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375788 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375680 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-oauth-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375788 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375705 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrnmm\" (UniqueName: \"kubernetes.io/projected/15b08258-5a2f-4a27-b95b-6055abad1241-kube-api-access-wrnmm\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375788 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-service-ca\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375912 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375807 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-trusted-ca-bundle\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375912 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375825 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-oauth-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.375912 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.375847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.476902 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.476878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-console-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.476910 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-oauth-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.476954 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrnmm\" (UniqueName: \"kubernetes.io/projected/15b08258-5a2f-4a27-b95b-6055abad1241-kube-api-access-wrnmm\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477011 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-service-ca\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477143 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477028 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-trusted-ca-bundle\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477143 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477055 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-oauth-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477143 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477652 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477607 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-oauth-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477764 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477747 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-console-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.477830 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.477809 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-service-ca\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.478036 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.478009 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/15b08258-5a2f-4a27-b95b-6055abad1241-trusted-ca-bundle\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.479621 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.479592 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-oauth-config\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.479744 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.479728 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/15b08258-5a2f-4a27-b95b-6055abad1241-console-serving-cert\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.486269 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.486240 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrnmm\" (UniqueName: \"kubernetes.io/projected/15b08258-5a2f-4a27-b95b-6055abad1241-kube-api-access-wrnmm\") pod \"console-5f47cc9f65-8474r\" (UID: \"15b08258-5a2f-4a27-b95b-6055abad1241\") " pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.648467 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.648437 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:06.803544 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:06.803518 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f47cc9f65-8474r"] Jun 14 19:22:06.805759 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:22:06.805730 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15b08258_5a2f_4a27_b95b_6055abad1241.slice/crio-3563e2c09129242640ee7b026c174c8eb49078884ddbf26aaad32d81430832d1 WatchSource:0}: Error finding container 3563e2c09129242640ee7b026c174c8eb49078884ddbf26aaad32d81430832d1: Status 404 returned error can't find the container with id 3563e2c09129242640ee7b026c174c8eb49078884ddbf26aaad32d81430832d1 Jun 14 19:22:07.161285 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:07.161248 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cc9f65-8474r" event={"ID":"15b08258-5a2f-4a27-b95b-6055abad1241","Type":"ContainerStarted","Data":"a482c0f7b990ce885b70466de303bd1a3a487de6b6f6aea7cf4a62cc18c2c45e"} Jun 14 19:22:07.161454 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:07.161309 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f47cc9f65-8474r" event={"ID":"15b08258-5a2f-4a27-b95b-6055abad1241","Type":"ContainerStarted","Data":"3563e2c09129242640ee7b026c174c8eb49078884ddbf26aaad32d81430832d1"} Jun 14 19:22:07.179190 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:07.179108 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f47cc9f65-8474r" podStartSLOduration=1.179094364 podStartE2EDuration="1.179094364s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:22:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:22:07.178162675 +0000 UTC m=+654.778112457" watchObservedRunningTime="2026-06-14 19:22:07.179094364 +0000 UTC m=+654.779044145" Jun 14 19:22:16.648739 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:16.648702 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:16.649136 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:16.648793 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:16.653538 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:16.653514 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:17.208697 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:17.208672 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f47cc9f65-8474r" Jun 14 19:22:19.445197 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.445158 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:22:19.480906 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.480878 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:22:19.481052 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.480915 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:22:19.481052 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.480983 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.483567 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.483547 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 14 19:22:19.579014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.578989 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gb62\" (UniqueName: \"kubernetes.io/projected/cbe5e66e-64a7-4e3a-a478-100aa25621ae-kube-api-access-8gb62\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.579155 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.579025 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cbe5e66e-64a7-4e3a-a478-100aa25621ae-config-file\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.680189 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.680163 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gb62\" (UniqueName: \"kubernetes.io/projected/cbe5e66e-64a7-4e3a-a478-100aa25621ae-kube-api-access-8gb62\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.680292 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.680207 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cbe5e66e-64a7-4e3a-a478-100aa25621ae-config-file\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.680857 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.680842 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/cbe5e66e-64a7-4e3a-a478-100aa25621ae-config-file\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.688026 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.688007 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gb62\" (UniqueName: \"kubernetes.io/projected/cbe5e66e-64a7-4e3a-a478-100aa25621ae-kube-api-access-8gb62\") pod \"limitador-limitador-54d6d988d5-nmrzx\" (UID: \"cbe5e66e-64a7-4e3a-a478-100aa25621ae\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.791252 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.791172 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:19.917178 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:19.917153 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:22:19.919972 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:22:19.919942 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbe5e66e_64a7_4e3a_a478_100aa25621ae.slice/crio-87eabac525997421d76105324c104983ef0faddcecaa1e9d268c7be8803308ce WatchSource:0}: Error finding container 87eabac525997421d76105324c104983ef0faddcecaa1e9d268c7be8803308ce: Status 404 returned error can't find the container with id 87eabac525997421d76105324c104983ef0faddcecaa1e9d268c7be8803308ce Jun 14 19:22:20.219593 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:20.219560 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" event={"ID":"cbe5e66e-64a7-4e3a-a478-100aa25621ae","Type":"ContainerStarted","Data":"87eabac525997421d76105324c104983ef0faddcecaa1e9d268c7be8803308ce"} Jun 14 19:22:23.234548 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:23.234512 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" event={"ID":"cbe5e66e-64a7-4e3a-a478-100aa25621ae","Type":"ContainerStarted","Data":"24631f594acaf77b080a5645e85fed9c0f94fa08fc3f2b508799bd36d02257a1"} Jun 14 19:22:23.235055 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:23.234592 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:22:23.254623 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:23.254576 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" podStartSLOduration=1.871058426 podStartE2EDuration="4.254561775s" totalImagesPullingTime="2.383503349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:22:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:22:23.253116006 +0000 UTC m=+670.853065786" watchObservedRunningTime="2026-06-14 19:22:23.254561775 +0000 UTC m=+670.854511605" Jun 14 19:22:34.239869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:22:34.239834 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-nmrzx" Jun 14 19:24:19.804012 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.803973 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-zz7r5"] Jun 14 19:24:19.807132 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.807114 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:19.809737 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.809721 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 14 19:24:19.809737 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.809729 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-hvrk2\"" Jun 14 19:24:19.817457 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.817438 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-zz7r5"] Jun 14 19:24:19.884118 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.884089 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:24:19.920606 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:19.920574 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpvrf\" (UniqueName: \"kubernetes.io/projected/28c8f641-a87e-4393-92b3-ef0441cbb512-kube-api-access-zpvrf\") pod \"payload-pre-processing-579785bc64-zz7r5\" (UID: \"28c8f641-a87e-4393-92b3-ef0441cbb512\") " pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:20.021442 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.021410 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpvrf\" (UniqueName: \"kubernetes.io/projected/28c8f641-a87e-4393-92b3-ef0441cbb512-kube-api-access-zpvrf\") pod \"payload-pre-processing-579785bc64-zz7r5\" (UID: \"28c8f641-a87e-4393-92b3-ef0441cbb512\") " pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:20.029375 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.029348 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpvrf\" (UniqueName: \"kubernetes.io/projected/28c8f641-a87e-4393-92b3-ef0441cbb512-kube-api-access-zpvrf\") pod \"payload-pre-processing-579785bc64-zz7r5\" (UID: \"28c8f641-a87e-4393-92b3-ef0441cbb512\") " pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:20.117326 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.117302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:20.242760 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.242620 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-579785bc64-zz7r5"] Jun 14 19:24:20.244975 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:24:20.244934 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28c8f641_a87e_4393_92b3_ef0441cbb512.slice/crio-42976d19de4c95b675f912f0e1aff70ba17435d6089556068038ebd0d2316c1b WatchSource:0}: Error finding container 42976d19de4c95b675f912f0e1aff70ba17435d6089556068038ebd0d2316c1b: Status 404 returned error can't find the container with id 42976d19de4c95b675f912f0e1aff70ba17435d6089556068038ebd0d2316c1b Jun 14 19:24:20.246808 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.246788 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 14 19:24:20.731014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:20.730973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" event={"ID":"28c8f641-a87e-4393-92b3-ef0441cbb512","Type":"ContainerStarted","Data":"42976d19de4c95b675f912f0e1aff70ba17435d6089556068038ebd0d2316c1b"} Jun 14 19:24:23.770793 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:23.770749 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" event={"ID":"28c8f641-a87e-4393-92b3-ef0441cbb512","Type":"ContainerStarted","Data":"807809f4e4ed96fe88232056b7c60efc4128d2426017d919c2216b6067c0339a"} Jun 14 19:24:23.771185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:23.770929 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:24:23.786596 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:23.786545 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" podStartSLOduration=2.018701515 podStartE2EDuration="4.786523775s" totalImagesPullingTime="2.76782226s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:24:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:24:23.784896756 +0000 UTC m=+791.384846537" watchObservedRunningTime="2026-06-14 19:24:23.786523775 +0000 UTC m=+791.386473559" Jun 14 19:24:27.790072 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.790038 2567 generic.go:356] "Generic (PLEG): container finished" podID="339c850a-81ed-487a-a976-f13b3324821c" containerID="9ea4fda1f39e7ce1cda751106b8ccad63088ca0b882e502480caff11eb951e61" exitCode=0 Jun 14 19:24:27.790485 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.790118 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" event={"ID":"339c850a-81ed-487a-a976-f13b3324821c","Type":"ContainerDied","Data":"9ea4fda1f39e7ce1cda751106b8ccad63088ca0b882e502480caff11eb951e61"} Jun 14 19:24:27.790485 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.790460 2567 scope.go:117] "RemoveContainer" containerID="9ea4fda1f39e7ce1cda751106b8ccad63088ca0b882e502480caff11eb951e61" Jun 14 19:24:27.791401 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.791371 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:24:27.791499 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.791454 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:24:27.791558 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:27.791517 2567 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jun 14 19:24:28.796422 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:28.796388 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" event={"ID":"339c850a-81ed-487a-a976-f13b3324821c","Type":"ContainerStarted","Data":"bd6ad57b753720e772aa99ad72d35f2188de5d1bb577e475446b6753d6fa7367"} Jun 14 19:24:29.441372 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:29.441281 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:24:29.442007 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:29.441541 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-v6kg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jun 14 19:24:29.442007 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:29.441716 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" podUID="339c850a-81ed-487a-a976-f13b3324821c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jun 14 19:24:30.441340 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:30.441297 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-v6kg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jun 14 19:24:30.441886 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:30.441373 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" podUID="339c850a-81ed-487a-a976-f13b3324821c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jun 14 19:24:31.441723 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:31.441685 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-v6kg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jun 14 19:24:31.442297 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:31.442264 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" podUID="339c850a-81ed-487a-a976-f13b3324821c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jun 14 19:24:32.441951 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:32.441917 2567 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-v6kg2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jun 14 19:24:32.442339 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:32.441972 2567 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" podUID="339c850a-81ed-487a-a976-f13b3324821c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jun 14 19:24:33.445346 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:33.445311 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:24:33.445823 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:33.445666 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:24:33.446413 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:33.446396 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-v6kg2" Jun 14 19:24:34.778238 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:24:34.778207 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-579785bc64-zz7r5" Jun 14 19:25:29.714888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.714858 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt"] Jun 14 19:25:29.718344 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.718323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.720912 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.720893 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9b7tg\"" Jun 14 19:25:29.721045 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.720912 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 14 19:25:29.721045 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.720893 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 14 19:25:29.722036 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.722021 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 14 19:25:29.729438 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.729418 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt"] Jun 14 19:25:29.790448 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790423 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.790536 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790466 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.790536 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790484 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.790536 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60126e12-c775-41f0-b045-710db8d5e4b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.790662 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790562 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfvr4\" (UniqueName: \"kubernetes.io/projected/60126e12-c775-41f0-b045-710db8d5e4b1-kube-api-access-vfvr4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.790662 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.790606 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892025 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.891993 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892163 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892052 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892163 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892109 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892163 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892139 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892282 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892176 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60126e12-c775-41f0-b045-710db8d5e4b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892332 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892309 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfvr4\" (UniqueName: \"kubernetes.io/projected/60126e12-c775-41f0-b045-710db8d5e4b1-kube-api-access-vfvr4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892456 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892435 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892526 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892463 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.892669 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.892519 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.894485 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.894464 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/60126e12-c775-41f0-b045-710db8d5e4b1-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.894582 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.894568 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60126e12-c775-41f0-b045-710db8d5e4b1-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:29.899745 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:29.899727 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfvr4\" (UniqueName: \"kubernetes.io/projected/60126e12-c775-41f0-b045-710db8d5e4b1-kube-api-access-vfvr4\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt\" (UID: \"60126e12-c775-41f0-b045-710db8d5e4b1\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:30.022249 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:30.022187 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:25:30.029687 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:30.029664 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:25:30.160235 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:30.160178 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt"] Jun 14 19:25:30.163073 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:25:30.163051 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60126e12_c775_41f0_b045_710db8d5e4b1.slice/crio-42405f0a82fbfb43045b6d6ab12ea156ba534dfe6b115ed50d6464a176a2fea9 WatchSource:0}: Error finding container 42405f0a82fbfb43045b6d6ab12ea156ba534dfe6b115ed50d6464a176a2fea9: Status 404 returned error can't find the container with id 42405f0a82fbfb43045b6d6ab12ea156ba534dfe6b115ed50d6464a176a2fea9 Jun 14 19:25:31.060904 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:31.060865 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" event={"ID":"60126e12-c775-41f0-b045-710db8d5e4b1","Type":"ContainerStarted","Data":"42405f0a82fbfb43045b6d6ab12ea156ba534dfe6b115ed50d6464a176a2fea9"} Jun 14 19:25:34.075693 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:34.075658 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" event={"ID":"60126e12-c775-41f0-b045-710db8d5e4b1","Type":"ContainerStarted","Data":"740debaa9025c4c2891eb03d691721208ae3d7714d9216833acb8abd8f6f061c"} Jun 14 19:25:56.910282 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:56.910249 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl"] Jun 14 19:25:56.926917 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:56.926889 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl"] Jun 14 19:25:56.927054 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:56.926978 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:56.930241 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:56.930218 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 14 19:25:57.019787 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.019758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.019954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.019807 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.019954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.019884 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039d21c7-5511-482c-af2f-d143677c0f32-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.019954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.019935 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.020079 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.020000 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.020121 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.020099 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pql2m\" (UniqueName: \"kubernetes.io/projected/039d21c7-5511-482c-af2f-d143677c0f32-kube-api-access-pql2m\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121377 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pql2m\" (UniqueName: \"kubernetes.io/projected/039d21c7-5511-482c-af2f-d143677c0f32-kube-api-access-pql2m\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121521 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121389 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121521 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121503 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121731 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121536 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039d21c7-5511-482c-af2f-d143677c0f32-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121731 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121731 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121603 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121888 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121790 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.121955 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121938 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.122013 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.121983 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.123849 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.123830 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/039d21c7-5511-482c-af2f-d143677c0f32-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.124234 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.124214 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/039d21c7-5511-482c-af2f-d143677c0f32-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.129496 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.129474 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pql2m\" (UniqueName: \"kubernetes.io/projected/039d21c7-5511-482c-af2f-d143677c0f32-kube-api-access-pql2m\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b2wxl\" (UID: \"039d21c7-5511-482c-af2f-d143677c0f32\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.263486 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.263408 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:25:57.388050 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.387992 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl"] Jun 14 19:25:57.390157 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:25:57.390127 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod039d21c7_5511_482c_af2f_d143677c0f32.slice/crio-599fccf08c976064d98e997cdc539f476df004c73d51d59f3869d50e773efd55 WatchSource:0}: Error finding container 599fccf08c976064d98e997cdc539f476df004c73d51d59f3869d50e773efd55: Status 404 returned error can't find the container with id 599fccf08c976064d98e997cdc539f476df004c73d51d59f3869d50e773efd55 Jun 14 19:25:57.618941 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:57.618906 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:25:58.184059 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:58.184025 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" event={"ID":"039d21c7-5511-482c-af2f-d143677c0f32","Type":"ContainerStarted","Data":"1eeead5ff156d5b5663192a4193ad5c435ba049818ae1015fbd7d51f189e6eaa"} Jun 14 19:25:58.184435 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:58.184079 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" event={"ID":"039d21c7-5511-482c-af2f-d143677c0f32","Type":"ContainerStarted","Data":"599fccf08c976064d98e997cdc539f476df004c73d51d59f3869d50e773efd55"} Jun 14 19:25:59.189455 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:59.189420 2567 generic.go:356] "Generic (PLEG): container finished" podID="039d21c7-5511-482c-af2f-d143677c0f32" containerID="1eeead5ff156d5b5663192a4193ad5c435ba049818ae1015fbd7d51f189e6eaa" exitCode=0 Jun 14 19:25:59.189952 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:25:59.189485 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" event={"ID":"039d21c7-5511-482c-af2f-d143677c0f32","Type":"ContainerDied","Data":"1eeead5ff156d5b5663192a4193ad5c435ba049818ae1015fbd7d51f189e6eaa"} Jun 14 19:26:01.199770 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:01.199733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" event={"ID":"039d21c7-5511-482c-af2f-d143677c0f32","Type":"ContainerStarted","Data":"f6e12738f3605e8031dc1cac82905c53922e4e9078f6dc501c054e89a4a491f4"} Jun 14 19:26:01.218688 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:01.218623 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" podStartSLOduration=2.894482337 podStartE2EDuration="5.218608597s" totalImagesPullingTime="1.039931996s" totalInitContainerRuntime="1.284194264s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:25:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:26:01.217191616 +0000 UTC m=+888.817141417" watchObservedRunningTime="2026-06-14 19:26:01.218608597 +0000 UTC m=+888.818558377" Jun 14 19:26:07.263980 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:07.263947 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:26:07.264374 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:07.264021 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:26:07.276582 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:07.276559 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:26:08.240933 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:08.240902 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b2wxl" Jun 14 19:26:20.434870 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.434837 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2"] Jun 14 19:26:20.438833 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.438817 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.441294 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.441275 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 14 19:26:20.447954 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.447933 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2"] Jun 14 19:26:20.521199 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521175 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.521322 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521228 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57bb5aa9-2880-47b6-a663-89f1d524c193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.521322 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521271 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.521322 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521288 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8xh2\" (UniqueName: \"kubernetes.io/projected/57bb5aa9-2880-47b6-a663-89f1d524c193-kube-api-access-l8xh2\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.521322 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521308 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.521492 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.521335 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622517 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622541 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57bb5aa9-2880-47b6-a663-89f1d524c193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622572 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622590 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8xh2\" (UniqueName: \"kubernetes.io/projected/57bb5aa9-2880-47b6-a663-89f1d524c193-kube-api-access-l8xh2\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622608 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622677 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622627 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.622986 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.622967 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.623034 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.623001 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.623070 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.623043 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.624965 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.624942 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/57bb5aa9-2880-47b6-a663-89f1d524c193-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.625166 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.625150 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57bb5aa9-2880-47b6-a663-89f1d524c193-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.630380 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.630359 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8xh2\" (UniqueName: \"kubernetes.io/projected/57bb5aa9-2880-47b6-a663-89f1d524c193-kube-api-access-l8xh2\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2\" (UID: \"57bb5aa9-2880-47b6-a663-89f1d524c193\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.749777 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.749672 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:20.879755 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:20.879731 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2"] Jun 14 19:26:20.882025 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:26:20.882000 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57bb5aa9_2880_47b6_a663_89f1d524c193.slice/crio-89d6c9d3a5810eb399e7d95e103b5ec6f6822cb3f4dee363e8ba0f60d31dd050 WatchSource:0}: Error finding container 89d6c9d3a5810eb399e7d95e103b5ec6f6822cb3f4dee363e8ba0f60d31dd050: Status 404 returned error can't find the container with id 89d6c9d3a5810eb399e7d95e103b5ec6f6822cb3f4dee363e8ba0f60d31dd050 Jun 14 19:26:21.219774 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:21.219740 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:26:21.287339 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:21.287301 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" event={"ID":"57bb5aa9-2880-47b6-a663-89f1d524c193","Type":"ContainerStarted","Data":"1bd961aed2cc434677110f824c26ca92d1b03cba02451c19c8e117dd682501f6"} Jun 14 19:26:21.287521 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:21.287363 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" event={"ID":"57bb5aa9-2880-47b6-a663-89f1d524c193","Type":"ContainerStarted","Data":"89d6c9d3a5810eb399e7d95e103b5ec6f6822cb3f4dee363e8ba0f60d31dd050"} Jun 14 19:26:25.306347 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:25.306320 2567 generic.go:356] "Generic (PLEG): container finished" podID="57bb5aa9-2880-47b6-a663-89f1d524c193" containerID="1bd961aed2cc434677110f824c26ca92d1b03cba02451c19c8e117dd682501f6" exitCode=0 Jun 14 19:26:25.306714 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:25.306399 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" event={"ID":"57bb5aa9-2880-47b6-a663-89f1d524c193","Type":"ContainerDied","Data":"1bd961aed2cc434677110f824c26ca92d1b03cba02451c19c8e117dd682501f6"} Jun 14 19:26:25.307957 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:25.307935 2567 generic.go:356] "Generic (PLEG): container finished" podID="60126e12-c775-41f0-b045-710db8d5e4b1" containerID="740debaa9025c4c2891eb03d691721208ae3d7714d9216833acb8abd8f6f061c" exitCode=0 Jun 14 19:26:25.308045 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:25.307990 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" event={"ID":"60126e12-c775-41f0-b045-710db8d5e4b1","Type":"ContainerDied","Data":"740debaa9025c4c2891eb03d691721208ae3d7714d9216833acb8abd8f6f061c"} Jun 14 19:26:26.312148 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.312113 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s"] Jun 14 19:26:26.315894 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.315861 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" event={"ID":"60126e12-c775-41f0-b045-710db8d5e4b1","Type":"ContainerStarted","Data":"cd9fe006ff9a346cada73f56ec30017ab66ffa18fb4f768bfc637ebd2add8233"} Jun 14 19:26:26.316033 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.315960 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.317894 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.317863 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" event={"ID":"57bb5aa9-2880-47b6-a663-89f1d524c193","Type":"ContainerStarted","Data":"522187e36bb514b78a5ca2dba2f5dddc91fa3195654013c40150c655bb4c03b4"} Jun 14 19:26:26.318520 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.318422 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 14 19:26:26.325274 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.325252 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s"] Jun 14 19:26:26.337707 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.337664 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" podStartSLOduration=2.168328228 podStartE2EDuration="57.337618225s" totalImagesPullingTime="3.762171205s" totalInitContainerRuntime="51.407118792s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:25:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:26:26.335883801 +0000 UTC m=+913.935833582" watchObservedRunningTime="2026-06-14 19:26:26.337618225 +0000 UTC m=+913.937568007" Jun 14 19:26:26.356510 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.356464 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" podStartSLOduration=2.028534239 podStartE2EDuration="6.35645075s" totalImagesPullingTime="193.127137ms" totalInitContainerRuntime="4.134789374s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:26:26.354989584 +0000 UTC m=+913.954939379" watchObservedRunningTime="2026-06-14 19:26:26.35645075 +0000 UTC m=+913.956400560" Jun 14 19:26:26.468882 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.468845 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.469054 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.468911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.469147 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.469089 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.469147 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.469128 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8188dee3-562a-4d63-a787-629c4227f35b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.469260 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.469161 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.469317 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.469272 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jz5q\" (UniqueName: \"kubernetes.io/projected/8188dee3-562a-4d63-a787-629c4227f35b-kube-api-access-2jz5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570109 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570022 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jz5q\" (UniqueName: \"kubernetes.io/projected/8188dee3-562a-4d63-a787-629c4227f35b-kube-api-access-2jz5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570109 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570088 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570117 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570137 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570156 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8188dee3-562a-4d63-a787-629c4227f35b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570173 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570558 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570539 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570614 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570591 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.570706 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.570664 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.572983 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.572954 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8188dee3-562a-4d63-a787-629c4227f35b-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.573089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.572957 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8188dee3-562a-4d63-a787-629c4227f35b-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.577470 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.577447 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jz5q\" (UniqueName: \"kubernetes.io/projected/8188dee3-562a-4d63-a787-629c4227f35b-kube-api-access-2jz5q\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s\" (UID: \"8188dee3-562a-4d63-a787-629c4227f35b\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.627475 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.627447 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:26.755839 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:26.755816 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s"] Jun 14 19:26:26.758099 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:26:26.758061 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8188dee3_562a_4d63_a787_629c4227f35b.slice/crio-0be5b1d41ff5b04168519bfdd6c7bb19069d2aea8b756da22e5ac787cecae0e4 WatchSource:0}: Error finding container 0be5b1d41ff5b04168519bfdd6c7bb19069d2aea8b756da22e5ac787cecae0e4: Status 404 returned error can't find the container with id 0be5b1d41ff5b04168519bfdd6c7bb19069d2aea8b756da22e5ac787cecae0e4 Jun 14 19:26:27.217883 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:27.217847 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:26:27.324621 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:27.324582 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" event={"ID":"8188dee3-562a-4d63-a787-629c4227f35b","Type":"ContainerStarted","Data":"13b94cbcd7ece06fe771d56664b15afb7ce9581ff673fea4cde0169ea872a754"} Jun 14 19:26:27.325087 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:27.324659 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" event={"ID":"8188dee3-562a-4d63-a787-629c4227f35b","Type":"ContainerStarted","Data":"0be5b1d41ff5b04168519bfdd6c7bb19069d2aea8b756da22e5ac787cecae0e4"} Jun 14 19:26:30.030411 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.030382 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:26:30.030788 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.030442 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:26:30.043132 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.043112 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:26:30.348796 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.348762 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt" Jun 14 19:26:30.749934 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.749854 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:30.749934 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.749918 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:30.762214 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:30.762188 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:31.342725 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:31.342694 2567 generic.go:356] "Generic (PLEG): container finished" podID="8188dee3-562a-4d63-a787-629c4227f35b" containerID="13b94cbcd7ece06fe771d56664b15afb7ce9581ff673fea4cde0169ea872a754" exitCode=0 Jun 14 19:26:31.343097 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:31.342765 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" event={"ID":"8188dee3-562a-4d63-a787-629c4227f35b","Type":"ContainerDied","Data":"13b94cbcd7ece06fe771d56664b15afb7ce9581ff673fea4cde0169ea872a754"} Jun 14 19:26:31.354335 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:31.354316 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2" Jun 14 19:26:32.349875 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:32.349841 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" event={"ID":"8188dee3-562a-4d63-a787-629c4227f35b","Type":"ContainerStarted","Data":"8ad57fc9bf1eae8e7bfb846a8b5908a18d6a68912510469b5df4dbc341901c54"} Jun 14 19:26:32.368611 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:32.368566 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" podStartSLOduration=2.113877377 podStartE2EDuration="6.368553767s" totalImagesPullingTime="184.642974ms" totalInitContainerRuntime="4.070033416s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:26:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:26:32.367861236 +0000 UTC m=+919.967811018" watchObservedRunningTime="2026-06-14 19:26:32.368553767 +0000 UTC m=+919.968503547" Jun 14 19:26:36.628386 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:36.628344 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:36.628802 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:36.628450 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:36.640780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:36.640754 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:26:37.388780 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:26:37.388752 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s" Jun 14 19:27:04.433613 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.433581 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz"] Jun 14 19:27:04.437296 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.437278 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.439706 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.439681 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 14 19:27:04.446756 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.446732 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz"] Jun 14 19:27:04.500799 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500777 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlnwj\" (UniqueName: \"kubernetes.io/projected/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kube-api-access-hlnwj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.500923 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500816 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.500923 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500894 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.500923 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500916 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.501026 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.501026 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.500962 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ea9e09-a291-49dd-92a7-06fd2c208c1d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602099 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602051 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602280 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602280 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602234 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602280 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602461 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602286 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ea9e09-a291-49dd-92a7-06fd2c208c1d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602461 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlnwj\" (UniqueName: \"kubernetes.io/projected/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kube-api-access-hlnwj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.602682 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602658 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.603020 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.602991 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.603333 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.603308 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.605529 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.605509 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/16ea9e09-a291-49dd-92a7-06fd2c208c1d-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.605627 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.605602 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/16ea9e09-a291-49dd-92a7-06fd2c208c1d-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.610415 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.610371 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlnwj\" (UniqueName: \"kubernetes.io/projected/16ea9e09-a291-49dd-92a7-06fd2c208c1d-kube-api-access-hlnwj\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-vjclz\" (UID: \"16ea9e09-a291-49dd-92a7-06fd2c208c1d\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.747535 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.747442 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:04.881668 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:04.881644 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz"] Jun 14 19:27:05.021014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:05.020921 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:27:05.506256 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:05.506221 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" event={"ID":"16ea9e09-a291-49dd-92a7-06fd2c208c1d","Type":"ContainerStarted","Data":"5e548d498d906608c3e9053315079ac239e1d5dfb529469c6e6e7872cfd02299"} Jun 14 19:27:05.506618 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:05.506270 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" event={"ID":"16ea9e09-a291-49dd-92a7-06fd2c208c1d","Type":"ContainerStarted","Data":"ec5a07aed0844e9d39f5bfa25c3f4e5ebcab7388649e99a84db2507a98e55310"} Jun 14 19:27:06.512851 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:06.512815 2567 generic.go:356] "Generic (PLEG): container finished" podID="16ea9e09-a291-49dd-92a7-06fd2c208c1d" containerID="5e548d498d906608c3e9053315079ac239e1d5dfb529469c6e6e7872cfd02299" exitCode=0 Jun 14 19:27:06.512851 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:06.512852 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" event={"ID":"16ea9e09-a291-49dd-92a7-06fd2c208c1d","Type":"ContainerDied","Data":"5e548d498d906608c3e9053315079ac239e1d5dfb529469c6e6e7872cfd02299"} Jun 14 19:27:07.519309 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:07.519274 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" event={"ID":"16ea9e09-a291-49dd-92a7-06fd2c208c1d","Type":"ContainerStarted","Data":"ba87b8c6060105245cebddac1a148fe73662e31c5334949e2e0b14f97f018e04"} Jun 14 19:27:07.539217 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:07.539173 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" podStartSLOduration=2.03429206 podStartE2EDuration="3.539157849s" totalImagesPullingTime="160.884467ms" totalInitContainerRuntime="1.343981322s" isStatefulPod=false podCreationTimestamp="2026-06-14 19:27:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:27:07.536918624 +0000 UTC m=+955.136868424" watchObservedRunningTime="2026-06-14 19:27:07.539157849 +0000 UTC m=+955.139107629" Jun 14 19:27:09.615163 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:09.615130 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-nmrzx"] Jun 14 19:27:14.747741 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:14.747702 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:14.748185 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:14.747800 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:14.760427 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:14.760397 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:27:15.566950 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:27:15.566925 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-vjclz" Jun 14 19:32:58.513799 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:32:58.513771 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7d7df895cb-s99dc_1b5c4cfa-a53f-40b3-b5d5-4b66de14c379/nginx/0.log" Jun 14 19:32:59.657617 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:32:59.657587 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f8bf57587-zhrh4_81448164-63f1-49d6-93b3-478bdc993f61/manager/0.log" Jun 14 19:32:59.667967 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:32:59.667947 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f8bf57587-zhrh4_81448164-63f1-49d6-93b3-478bdc993f61/copy-manifests/0.log" Jun 14 19:33:01.242821 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:01.242790 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5vpll_3a07c031-874d-40c5-8c1e-6ea42197a114/manager/0.log" Jun 14 19:33:01.356076 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:01.356046 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-584xb_2f44d420-3920-476a-b44f-806e7c388e21/kuadrant-console-plugin/0.log" Jun 14 19:33:01.465258 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:01.465228 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2xsps_2491c3cf-5113-41df-987c-1eab9242dfb4/registry-server/0.log" Jun 14 19:33:01.687871 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:01.687839 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nmrzx_cbe5e66e-64a7-4e3a-a478-100aa25621ae/limitador/0.log" Jun 14 19:33:01.801615 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:01.801584 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-97n5l_877bb89a-6022-4c8f-89e2-cc07d12009e2/manager/0.log" Jun 14 19:33:02.144396 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:02.144369 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795gc4vv_156d3807-3ca3-420f-aa1f-34dbd2e2f6bd/istio-proxy/0.log" Jun 14 19:33:02.597140 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:02.597112 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/1.log" Jun 14 19:33:02.707982 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:02.707954 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-zz7r5_28c8f641-a87e-4393-92b3-ef0441cbb512/payload-pre-processing/0.log" Jun 14 19:33:03.265937 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.265904 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt_60126e12-c775-41f0-b045-710db8d5e4b1/storage-initializer/0.log" Jun 14 19:33:03.273314 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.273287 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-gwjkt_60126e12-c775-41f0-b045-710db8d5e4b1/main/0.log" Jun 14 19:33:03.385274 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.385241 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-b2wxl_039d21c7-5511-482c-af2f-d143677c0f32/storage-initializer/0.log" Jun 14 19:33:03.392403 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.392377 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-b2wxl_039d21c7-5511-482c-af2f-d143677c0f32/main/0.log" Jun 14 19:33:03.506069 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.506040 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2_57bb5aa9-2880-47b6-a663-89f1d524c193/storage-initializer/0.log" Jun 14 19:33:03.513720 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.513697 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-67pl2_57bb5aa9-2880-47b6-a663-89f1d524c193/main/0.log" Jun 14 19:33:03.625392 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.625362 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s_8188dee3-562a-4d63-a787-629c4227f35b/main/0.log" Jun 14 19:33:03.633538 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.633516 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477cktf8s_8188dee3-562a-4d63-a787-629c4227f35b/storage-initializer/0.log" Jun 14 19:33:03.745980 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.745952 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-vjclz_16ea9e09-a291-49dd-92a7-06fd2c208c1d/storage-initializer/0.log" Jun 14 19:33:03.753256 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:03.753233 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-vjclz_16ea9e09-a291-49dd-92a7-06fd2c208c1d/main/0.log" Jun 14 19:33:10.851546 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:10.851519 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9zd6p_ca26259f-d764-4620-bf6d-be60ee15e22e/global-pull-secret-syncer/0.log" Jun 14 19:33:11.075419 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:11.075388 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-qgnw7_c89bece3-08c0-4010-88ee-3c98ee534fea/konnectivity-agent/0.log" Jun 14 19:33:11.175173 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:11.175090 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-135-103.ec2.internal_d515948fb06c638da248f4a0591134e9/haproxy/0.log" Jun 14 19:33:16.005047 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:16.005013 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-5vpll_3a07c031-874d-40c5-8c1e-6ea42197a114/manager/0.log" Jun 14 19:33:16.031380 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:16.031351 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-584xb_2f44d420-3920-476a-b44f-806e7c388e21/kuadrant-console-plugin/0.log" Jun 14 19:33:16.062220 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:16.062193 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-2xsps_2491c3cf-5113-41df-987c-1eab9242dfb4/registry-server/0.log" Jun 14 19:33:16.127735 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:16.127714 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-nmrzx_cbe5e66e-64a7-4e3a-a478-100aa25621ae/limitador/0.log" Jun 14 19:33:16.184148 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:16.184127 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-97n5l_877bb89a-6022-4c8f-89e2-cc07d12009e2/manager/0.log" Jun 14 19:33:18.303201 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.303170 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t48md_b7047cff-c0e7-4c86-acdd-81643ba4fd8f/node-exporter/0.log" Jun 14 19:33:18.331299 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.331267 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t48md_b7047cff-c0e7-4c86-acdd-81643ba4fd8f/kube-rbac-proxy/0.log" Jun 14 19:33:18.364364 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.364342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-t48md_b7047cff-c0e7-4c86-acdd-81643ba4fd8f/init-textfile/0.log" Jun 14 19:33:18.404405 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.404384 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-lk6vc_9e8b7536-57b4-4133-9e92-823910ebf461/kube-rbac-proxy-main/0.log" Jun 14 19:33:18.433619 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.433599 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-lk6vc_9e8b7536-57b4-4133-9e92-823910ebf461/kube-rbac-proxy-self/0.log" Jun 14 19:33:18.456734 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:18.456716 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-6b8d965dd5-lk6vc_9e8b7536-57b4-4133-9e92-823910ebf461/openshift-state-metrics/0.log" Jun 14 19:33:21.437374 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.437342 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f47cc9f65-8474r_15b08258-5a2f-4a27-b95b-6055abad1241/console/0.log" Jun 14 19:33:21.858142 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.858114 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84"] Jun 14 19:33:21.861338 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.861323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:21.863828 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.863800 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqf7v\"/\"kube-root-ca.crt\"" Jun 14 19:33:21.864899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.864882 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pqf7v\"/\"default-dockercfg-xdcnz\"" Jun 14 19:33:21.864999 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.864898 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pqf7v\"/\"openshift-service-ca.crt\"" Jun 14 19:33:21.868928 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.868910 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84"] Jun 14 19:33:21.962274 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.962251 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-sys\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:21.962402 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.962281 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksx8t\" (UniqueName: \"kubernetes.io/projected/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-kube-api-access-ksx8t\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:21.962402 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.962301 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-proc\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:21.962481 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.962418 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-podres\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:21.962532 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:21.962501 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-lib-modules\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063549 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063524 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-proc\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063711 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063600 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-podres\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063711 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063665 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-proc\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063711 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063699 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-lib-modules\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063866 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063742 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-sys\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063866 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063754 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-podres\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063866 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063767 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksx8t\" (UniqueName: \"kubernetes.io/projected/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-kube-api-access-ksx8t\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063866 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063787 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-lib-modules\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.063866 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.063798 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-sys\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.071305 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.071286 2567 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksx8t\" (UniqueName: \"kubernetes.io/projected/6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb-kube-api-access-ksx8t\") pod \"perf-node-gather-daemonset-kqp84\" (UID: \"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb\") " pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.172513 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.172462 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:22.302869 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:22.302845 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84"] Jun 14 19:33:22.305583 ip-10-0-135-103 kubenswrapper[2567]: W0614 19:33:22.305557 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6fe95ea9_0913_4b77_b2d9_bb334c7d0fbb.slice/crio-62509a3d45ce178c64906f49be00ec53f202ed105b80f6ccd4c4054abed159dd WatchSource:0}: Error finding container 62509a3d45ce178c64906f49be00ec53f202ed105b80f6ccd4c4054abed159dd: Status 404 returned error can't find the container with id 62509a3d45ce178c64906f49be00ec53f202ed105b80f6ccd4c4054abed159dd Jun 14 19:33:23.210602 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.210563 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" event={"ID":"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb","Type":"ContainerStarted","Data":"ea20ad2940e8666b23089978c2efd8e1e37cc0e3c84232ce2fa6fdaba915476a"} Jun 14 19:33:23.211089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.210613 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" event={"ID":"6fe95ea9-0913-4b77-b2d9-bb334c7d0fbb","Type":"ContainerStarted","Data":"62509a3d45ce178c64906f49be00ec53f202ed105b80f6ccd4c4054abed159dd"} Jun 14 19:33:23.211089 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.210667 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:23.226596 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.226554 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" podStartSLOduration=2.226541553 podStartE2EDuration="2.226541553s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-14 19:33:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-14 19:33:23.225764336 +0000 UTC m=+1330.825714135" watchObservedRunningTime="2026-06-14 19:33:23.226541553 +0000 UTC m=+1330.826491334" Jun 14 19:33:23.491910 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.491813 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9m4c_915d84aa-5f57-4116-955f-f02b9743be96/dns/0.log" Jun 14 19:33:23.513935 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.513909 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-q9m4c_915d84aa-5f57-4116-955f-f02b9743be96/kube-rbac-proxy/0.log" Jun 14 19:33:23.538463 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:23.538438 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-695qr_51d8c3c1-c3e3-4f68-9af5-a36bfcf3201e/dns-node-resolver/0.log" Jun 14 19:33:24.085211 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:24.085184 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c6fd6c8b4-zmz6j_f2ad0393-407d-438f-b98b-70758d12d005/registry/0.log" Jun 14 19:33:24.089014 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:24.088993 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c6fd6c8b4-zmz6j_f2ad0393-407d-438f-b98b-70758d12d005/registry/1.log" Jun 14 19:33:24.131900 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:24.131880 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-h6m4m_5655dd8a-5500-41b3-add1-0dc28a18292f/node-ca/0.log" Jun 14 19:33:25.090130 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:25.090091 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795gc4vv_156d3807-3ca3-420f-aa1f-34dbd2e2f6bd/istio-proxy/0.log" Jun 14 19:33:25.209724 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:25.209694 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/1.log" Jun 14 19:33:25.210128 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:25.210112 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/0.log" Jun 14 19:33:25.230127 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:25.230103 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-zz7r5_28c8f641-a87e-4393-92b3-ef0441cbb512/payload-pre-processing/0.log" Jun 14 19:33:25.871287 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:25.871225 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4p64b_e0be6651-035f-4092-935e-a0d95c666a67/serve-healthcheck-canary/0.log" Jun 14 19:33:26.420809 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:26.420780 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k4p54_43f5796f-b020-46d7-97b3-1f3ef75ffb0a/kube-rbac-proxy/0.log" Jun 14 19:33:26.444899 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:26.444866 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k4p54_43f5796f-b020-46d7-97b3-1f3ef75ffb0a/exporter/0.log" Jun 14 19:33:26.466015 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:26.465977 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-k4p54_43f5796f-b020-46d7-97b3-1f3ef75ffb0a/extractor/0.log" Jun 14 19:33:28.966857 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:28.966830 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7d7df895cb-s99dc_1b5c4cfa-a53f-40b3-b5d5-4b66de14c379/nginx/0.log" Jun 14 19:33:29.225550 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:29.225478 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pqf7v/perf-node-gather-daemonset-kqp84" Jun 14 19:33:29.343605 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:29.343536 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f8bf57587-zhrh4_81448164-63f1-49d6-93b3-478bdc993f61/manager/0.log" Jun 14 19:33:29.361727 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:29.361706 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7f8bf57587-zhrh4_81448164-63f1-49d6-93b3-478bdc993f61/copy-manifests/0.log" Jun 14 19:33:30.954868 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:30.954775 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-58b48f6fb9-6njft_21edea9d-49c6-4449-ba43-fbeabb4afb0f/manager/0.log" Jun 14 19:33:35.963370 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:35.963340 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795gc4vv_156d3807-3ca3-420f-aa1f-34dbd2e2f6bd/istio-proxy/0.log" Jun 14 19:33:36.062961 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:36.062936 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/1.log" Jun 14 19:33:36.063245 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:36.063229 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/0.log" Jun 14 19:33:36.074721 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:36.074701 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-zz7r5_28c8f641-a87e-4393-92b3-ef0441cbb512/payload-pre-processing/0.log" Jun 14 19:33:37.177440 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:37.177396 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795gc4vv_156d3807-3ca3-420f-aa1f-34dbd2e2f6bd/istio-proxy/0.log" Jun 14 19:33:37.274530 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:37.274496 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/1.log" Jun 14 19:33:37.274876 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:37.274859 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/0.log" Jun 14 19:33:37.286693 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:37.286673 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-zz7r5_28c8f641-a87e-4393-92b3-ef0441cbb512/payload-pre-processing/0.log" Jun 14 19:33:38.387540 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.387515 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/kube-multus-additional-cni-plugins/0.log" Jun 14 19:33:38.413058 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.413036 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/egress-router-binary-copy/0.log" Jun 14 19:33:38.435265 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.435244 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/cni-plugins/0.log" Jun 14 19:33:38.457975 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.457956 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/bond-cni-plugin/0.log" Jun 14 19:33:38.479351 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.479332 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/routeoverride-cni/0.log" Jun 14 19:33:38.501581 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.501562 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/whereabouts-cni-bincopy/0.log" Jun 14 19:33:38.525159 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.525138 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-pwt78_2ee89d03-f544-4b2f-8626-00b40bae5919/whereabouts-cni/0.log" Jun 14 19:33:38.592443 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.592421 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gmcf8_91037120-5701-470f-b5da-eef943987ef8/kube-multus/0.log" Jun 14 19:33:38.738358 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.738296 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nr9nh_3f5263ee-a22a-4417-9d14-ff9f08c65b1a/network-metrics-daemon/0.log" Jun 14 19:33:38.758464 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.758433 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-nr9nh_3f5263ee-a22a-4417-9d14-ff9f08c65b1a/kube-rbac-proxy/0.log" Jun 14 19:33:38.913517 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.913496 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-74759f9795gc4vv_156d3807-3ca3-420f-aa1f-34dbd2e2f6bd/istio-proxy/0.log" Jun 14 19:33:38.999088 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.999023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/1.log" Jun 14 19:33:38.999437 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:38.999423 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-v6kg2_339c850a-81ed-487a-a976-f13b3324821c/istio-proxy/0.log" Jun 14 19:33:39.008707 ip-10-0-135-103 kubenswrapper[2567]: I0614 19:33:39.008686 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-579785bc64-zz7r5_28c8f641-a87e-4393-92b3-ef0441cbb512/payload-pre-processing/0.log"