Jun 09 07:58:22.441157 ip-10-0-141-98 systemd[1]: Starting Kubernetes Kubelet... Jun 09 07:58:22.849933 ip-10-0-141-98 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 07:58:22.849933 ip-10-0-141-98 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 09 07:58:22.849933 ip-10-0-141-98 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 07:58:22.849933 ip-10-0-141-98 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 09 07:58:22.849933 ip-10-0-141-98 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 09 07:58:22.850921 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.850818 2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 09 07:58:22.854332 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854311 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.854332 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854330 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.854332 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854334 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854338 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854341 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854347 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854351 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854354 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854358 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854361 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854364 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854367 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854370 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854373 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854377 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854379 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854382 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854384 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854387 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854390 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854393 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.854432 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854396 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854398 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854406 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854409 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854412 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854415 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854418 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854420 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854423 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854425 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854428 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854430 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854434 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854436 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854439 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854444 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854448 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854451 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854454 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.854901 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854456 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854459 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854462 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854465 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854467 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854470 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854473 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854476 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854479 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854481 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854484 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854486 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854489 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854491 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854501 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854505 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854508 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854510 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854513 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854516 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.855374 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854518 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854521 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854523 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854526 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854529 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854531 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854534 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854537 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854539 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854542 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854545 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854548 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854552 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854554 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854557 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854560 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854562 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854565 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854567 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854570 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.855877 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854573 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854575 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854578 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854581 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854583 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854587 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854589 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854593 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854596 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854598 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854601 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854603 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854606 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854608 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854611 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854615 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854619 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854622 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.854625 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855346 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.856387 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855353 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855355 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855358 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855361 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855374 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855378 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855380 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855383 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855386 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855389 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855392 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855395 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855399 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855403 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855405 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855409 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855411 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855414 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855416 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855420 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.856914 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855423 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855426 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855428 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855431 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855434 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855436 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855439 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855441 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855444 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855446 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855449 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855453 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855456 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855459 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855462 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855465 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855469 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855472 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855475 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855478 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.857420 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855481 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855484 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855486 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855489 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855491 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855494 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855496 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855499 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855502 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855504 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855507 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855509 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855512 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855514 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855517 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855520 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855524 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855528 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855531 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.858025 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855533 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855536 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855538 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855541 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855544 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855546 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855549 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855551 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855554 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855557 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855559 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855563 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855565 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855568 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855570 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855573 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855575 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855578 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855580 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.858488 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855583 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855586 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855589 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855592 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855594 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855597 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855600 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855602 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855605 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855607 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855610 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855614 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855617 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855619 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855622 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855625 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855627 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855631 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855633 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.858976 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.855636 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856037 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856042 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856045 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856048 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856051 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856054 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856056 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856059 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856061 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856064 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856066 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856069 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856075 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856078 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856080 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856083 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856086 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856088 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856091 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.859463 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856093 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856096 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856098 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856101 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856104 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856106 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856109 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856112 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856115 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856117 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856120 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856122 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856125 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856128 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856130 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856135 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856139 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856143 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856146 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.859984 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856149 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856152 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856155 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856158 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856160 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856163 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856167 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856170 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856173 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856176 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856180 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856184 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856186 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856190 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856193 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856196 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856198 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856201 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856203 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856206 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.860458 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856209 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856211 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856214 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856216 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856219 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856221 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856224 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856226 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856229 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856231 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856234 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856238 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856241 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856245 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856248 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856251 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856253 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856256 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856260 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856263 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.861029 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856266 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856269 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856272 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856274 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856277 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856279 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856282 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856285 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856287 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856290 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856292 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856294 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856297 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856300 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856302 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856305 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856307 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856310 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856312 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856315 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.861512 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.856317 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856401 2573 flags.go:64] FLAG: --address="0.0.0.0" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856408 2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856415 2573 flags.go:64] FLAG: --anonymous-auth="true" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856419 2573 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856427 2573 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856430 2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856434 2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856439 2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856442 2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856446 2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856450 2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856453 2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856456 2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856459 2573 flags.go:64] FLAG: --cgroup-root="" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856462 2573 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856466 2573 flags.go:64] FLAG: --client-ca-file="" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856468 2573 flags.go:64] FLAG: --cloud-provider="external" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856472 2573 flags.go:64] FLAG: --cluster-dns="[]" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856477 2573 flags.go:64] FLAG: --cluster-domain="" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856480 2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856483 2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856487 2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856490 2573 flags.go:64] FLAG: --container-log-max-files="5" Jun 09 07:58:22.862049 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856494 2573 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856497 2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856500 2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856503 2573 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856506 2573 flags.go:64] FLAG: --contention-profiling="false" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856509 2573 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856512 2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856515 2573 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856518 2573 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856523 2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856526 2573 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856529 2573 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856532 2573 flags.go:64] FLAG: --enable-load-reader="false" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856535 2573 flags.go:64] FLAG: --enable-server="true" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856538 2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856543 2573 flags.go:64] FLAG: --event-burst="100" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856546 2573 flags.go:64] FLAG: --event-qps="50" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856549 2573 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856555 2573 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856558 2573 flags.go:64] FLAG: --eviction-hard="" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856562 2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856565 2573 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856568 2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856571 2573 flags.go:64] FLAG: --eviction-soft="" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856574 2573 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 09 07:58:22.862638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856577 2573 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856580 2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856583 2573 flags.go:64] FLAG: --experimental-mounter-path="" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856585 2573 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856588 2573 flags.go:64] FLAG: --fail-swap-on="true" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856591 2573 flags.go:64] FLAG: --feature-gates="" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856595 2573 flags.go:64] FLAG: --file-check-frequency="20s" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856598 2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856602 2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856605 2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856608 2573 flags.go:64] FLAG: --healthz-port="10248" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856611 2573 flags.go:64] FLAG: --help="false" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856614 2573 flags.go:64] FLAG: --hostname-override="ip-10-0-141-98.ec2.internal" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856617 2573 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856620 2573 flags.go:64] FLAG: --http-check-frequency="20s" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856623 2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856626 2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856630 2573 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856632 2573 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856635 2573 flags.go:64] FLAG: --image-service-endpoint="" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856638 2573 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856641 2573 flags.go:64] FLAG: --kube-api-burst="100" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856644 2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856647 2573 flags.go:64] FLAG: --kube-api-qps="50" Jun 09 07:58:22.863308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856650 2573 flags.go:64] FLAG: --kube-reserved="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856656 2573 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856659 2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856662 2573 flags.go:64] FLAG: --kubelet-cgroups="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856665 2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856668 2573 flags.go:64] FLAG: --lock-file="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856671 2573 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856674 2573 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856677 2573 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856683 2573 flags.go:64] FLAG: --log-json-split-stream="false" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856685 2573 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856688 2573 flags.go:64] FLAG: --log-text-split-stream="false" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856691 2573 flags.go:64] FLAG: --logging-format="text" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856694 2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856698 2573 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856700 2573 flags.go:64] FLAG: --manifest-url="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856703 2573 flags.go:64] FLAG: --manifest-url-header="" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856708 2573 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856711 2573 flags.go:64] FLAG: --max-open-files="1000000" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856715 2573 flags.go:64] FLAG: --max-pods="110" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856718 2573 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856721 2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856724 2573 flags.go:64] FLAG: --memory-manager-policy="None" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856727 2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856730 2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 09 07:58:22.863929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856733 2573 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856735 2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856743 2573 flags.go:64] FLAG: --node-status-max-images="50" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856746 2573 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856749 2573 flags.go:64] FLAG: --oom-score-adj="-999" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856752 2573 flags.go:64] FLAG: --pod-cidr="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856758 2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856764 2573 flags.go:64] FLAG: --pod-manifest-path="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856768 2573 flags.go:64] FLAG: --pod-max-pids="-1" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856773 2573 flags.go:64] FLAG: --pods-per-core="0" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856776 2573 flags.go:64] FLAG: --port="10250" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856779 2573 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856782 2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f30cec0eb9abfdbf" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856785 2573 flags.go:64] FLAG: --qos-reserved="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856788 2573 flags.go:64] FLAG: --read-only-port="10255" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856791 2573 flags.go:64] FLAG: --register-node="true" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856794 2573 flags.go:64] FLAG: --register-with-taints="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856798 2573 flags.go:64] FLAG: --registry-burst="10" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856801 2573 flags.go:64] FLAG: --registry-qps="5" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856804 2573 flags.go:64] FLAG: --reserved-cpus="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856807 2573 flags.go:64] FLAG: --reserved-memory="" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856811 2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856814 2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856817 2573 flags.go:64] FLAG: --rotate-certificates="false" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856820 2573 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 09 07:58:22.864547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856823 2573 flags.go:64] FLAG: --runonce="false" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856826 2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856829 2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856832 2573 flags.go:64] FLAG: --seccomp-default="false" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856834 2573 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856837 2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856840 2573 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856843 2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856846 2573 flags.go:64] FLAG: --storage-driver-password="root" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856850 2573 flags.go:64] FLAG: --storage-driver-secure="false" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856853 2573 flags.go:64] FLAG: --storage-driver-table="stats" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856856 2573 flags.go:64] FLAG: --storage-driver-user="root" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856858 2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856874 2573 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856877 2573 flags.go:64] FLAG: --system-cgroups="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856881 2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856889 2573 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856891 2573 flags.go:64] FLAG: --tls-cert-file="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856895 2573 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856899 2573 flags.go:64] FLAG: --tls-min-version="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856902 2573 flags.go:64] FLAG: --tls-private-key-file="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856905 2573 flags.go:64] FLAG: --topology-manager-policy="none" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856909 2573 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856912 2573 flags.go:64] FLAG: --topology-manager-scope="container" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856914 2573 flags.go:64] FLAG: --v="2" Jun 09 07:58:22.865185 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856919 2573 flags.go:64] FLAG: --version="false" Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856923 2573 flags.go:64] FLAG: --vmodule="" Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856927 2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.856930 2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857048 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857052 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857056 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857058 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857061 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857064 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857066 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857069 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857071 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857074 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857076 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857079 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857082 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857084 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857087 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857090 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.865796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857092 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857095 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857099 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857104 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857107 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857110 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857113 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857115 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857118 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857120 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857123 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857126 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857128 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857131 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857133 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857136 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857138 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857141 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857143 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.866320 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857146 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857149 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857151 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857154 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857156 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857159 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857161 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857164 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857167 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857169 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857171 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857174 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857176 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857179 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857182 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857187 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857190 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857193 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857196 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857198 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.866841 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857201 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857205 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857208 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857212 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857215 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857218 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857220 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857223 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857225 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857228 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857230 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857233 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857236 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857238 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857241 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857243 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857246 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857248 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857251 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857253 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.867389 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857256 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857258 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857261 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857264 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857266 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857269 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857271 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857275 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857278 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857281 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857284 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857286 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857288 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857291 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857293 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857296 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857298 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857302 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857305 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.867905 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857308 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857311 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857313 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857316 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.857319 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.857895 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.864915 2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.864932 2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.865745 2573 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.865753 2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865822 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865828 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865831 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865834 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865837 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.868370 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865840 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865843 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865846 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865848 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865851 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865854 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865856 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865878 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865882 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865885 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865888 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865891 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865894 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865897 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865899 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865906 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865909 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865912 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865915 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.868753 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865919 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865924 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865927 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865930 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865933 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865936 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865939 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865943 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865947 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865950 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865953 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865955 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865960 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865963 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865966 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865968 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865971 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865973 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865976 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.869294 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865979 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865981 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865989 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865992 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865995 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.865997 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866000 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866003 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866006 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866008 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866011 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866014 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866017 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866019 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866022 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866024 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866027 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866029 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866033 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866036 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.869767 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866038 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866041 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866043 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866046 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866049 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866052 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866054 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866057 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866059 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866062 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866065 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866067 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866070 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866072 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866074 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866084 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866087 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866090 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866093 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866096 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.870390 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866099 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866101 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866104 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866106 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866109 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866111 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866114 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866116 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866119 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866121 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866124 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866127 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866130 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866132 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866134 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.870911 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866137 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.866142 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866319 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866324 2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866327 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866330 2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866333 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866337 2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866340 2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866343 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866345 2573 feature_gate.go:328] unrecognized feature gate: Example Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866348 2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866350 2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866359 2573 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 09 07:58:22.871330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866361 2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866365 2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866367 2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866370 2573 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866373 2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866376 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866378 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866381 2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866383 2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866386 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866388 2573 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866391 2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866393 2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866396 2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866398 2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866401 2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866403 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866406 2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866409 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866411 2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 09 07:58:22.871741 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866414 2573 feature_gate.go:328] unrecognized feature gate: Example2 Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866416 2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866419 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866421 2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866423 2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866426 2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866430 2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866433 2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866436 2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866438 2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866441 2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866444 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866453 2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866456 2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866458 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866461 2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866463 2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866466 2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866468 2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 09 07:58:22.872354 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866471 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866473 2573 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866476 2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866478 2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866481 2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866483 2573 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866486 2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866488 2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866491 2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866493 2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866496 2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866498 2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866501 2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866503 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866505 2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866508 2573 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866512 2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866515 2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866518 2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 09 07:58:22.872826 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866521 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866524 2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866527 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866530 2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866533 2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866536 2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866538 2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866546 2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866549 2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866552 2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866554 2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866556 2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866559 2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866561 2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866564 2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866566 2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866569 2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866572 2573 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866574 2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866577 2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 09 07:58:22.873501 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866580 2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866582 2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866585 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866587 2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866590 2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866592 2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866595 2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866597 2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:22.866600 2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.866604 2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.866744 2573 server.go:962] "Client rotation is on, will bootstrap in background" Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.868814 2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.870035 2573 server.go:1019] "Starting client certificate rotation" Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.870157 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 07:58:22.874330 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.870221 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 09 07:58:22.874712 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.874630 2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 07:58:22.876237 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.876213 2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 09 07:58:22.887529 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.887506 2573 log.go:25] "Validated CRI v1 runtime API" Jun 09 07:58:22.890767 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.890748 2573 log.go:25] "Validated CRI v1 image API" Jun 09 07:58:22.891595 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.891579 2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 09 07:58:22.896500 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.896473 2573 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a7234fba-784f-4d11-a565-5e7713778e29:/dev/nvme0n1p4 b0f6bf9d-49f8-4a7f-a883-bb2208433a09:/dev/nvme0n1p3] Jun 09 07:58:22.896565 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.896500 2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 09 07:58:22.902242 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.902220 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 07:58:22.902476 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.902339 2573 manager.go:216] Machine: {Timestamp:2026-06-09 07:58:22.900233 +0000 UTC m=+0.340696969 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100565 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dd11c38f8c0fc4d5d7e0745aec3dd SystemUUID:ec2dd11c-38f8-c0fc-4d5d-7e0745aec3dd BootID:9d1a8efe-1f87-4649-9a25-99f727528632 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:7e:76:3a:ee:9d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:7e:76:3a:ee:9d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:62:3a:cc:b6:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 09 07:58:22.902476 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.902477 2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 09 07:58:22.902572 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.902565 2573 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 09 07:58:22.904815 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.904790 2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 09 07:58:22.905009 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.904825 2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-98.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 09 07:58:22.905512 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.905501 2573 topology_manager.go:138] "Creating topology manager with none policy" Jun 09 07:58:22.905544 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.905514 2573 container_manager_linux.go:309] "Creating device plugin manager" Jun 09 07:58:22.905544 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.905522 2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 07:58:22.906187 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.906176 2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 09 07:58:22.906954 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.906943 2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 09 07:58:22.909984 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.909972 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 09 07:58:22.910121 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910111 2573 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 09 07:58:22.910171 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910162 2573 kubelet.go:486] "Attempting to sync node with API server" Jun 09 07:58:22.910204 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910178 2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 09 07:58:22.910204 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910192 2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 09 07:58:22.910257 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910239 2573 kubelet.go:398] "Adding apiserver pod source" Jun 09 07:58:22.910257 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910249 2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 09 07:58:22.910639 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910626 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 07:58:22.910690 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.910647 2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 09 07:58:22.913220 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.913204 2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 09 07:58:22.916527 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.916507 2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 09 07:58:22.916637 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.916559 2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 09 07:58:22.917733 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917717 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917739 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917749 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917758 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917770 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917779 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917791 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 09 07:58:22.917808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917799 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917852 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917886 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917901 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917911 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917945 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 09 07:58:22.918132 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.917954 2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 09 07:58:22.921159 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.921136 2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-98.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 09 07:58:22.921276 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.921263 2573 server.go:1268] "Started kubelet" Jun 09 07:58:22.921428 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.921379 2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 09 07:58:22.921582 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.921552 2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 09 07:58:22.921673 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.921600 2573 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 09 07:58:22.921822 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.921770 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 09 07:58:22.921822 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.921774 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 09 07:58:22.922295 ip-10-0-141-98 systemd[1]: Started Kubernetes Kubelet. Jun 09 07:58:22.923914 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.923899 2573 server.go:310] "Adding debug handlers to kubelet server" Jun 09 07:58:22.926311 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.926294 2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 09 07:58:22.927715 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.926189 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-98.ec2.internal.18b75ab31977a345 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-98.ec2.internal,UID:ip-10-0-141-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-98.ec2.internal,},FirstTimestamp:2026-06-09 07:58:22.921237317 +0000 UTC m=+0.361701334,LastTimestamp:2026-06-09 07:58:22.921237317 +0000 UTC m=+0.361701334,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-98.ec2.internal,}" Jun 09 07:58:22.928635 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.928614 2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 09 07:58:22.931469 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.931455 2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 09 07:58:22.931531 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.931476 2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 09 07:58:22.931578 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.931565 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:22.932097 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932079 2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 09 07:58:22.932097 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932082 2573 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 09 07:58:22.932234 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932110 2573 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 09 07:58:22.932297 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932249 2573 reconstruct.go:97] "Volume reconstruction finished" Jun 09 07:58:22.932297 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932258 2573 reconciler.go:29] "Reconciler: start to sync state" Jun 09 07:58:22.932472 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932451 2573 factory.go:55] Registering systemd factory Jun 09 07:58:22.932472 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932475 2573 factory.go:223] Registration of the systemd container factory successfully Jun 09 07:58:22.932799 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932784 2573 factory.go:153] Registering CRI-O factory Jun 09 07:58:22.932799 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932799 2573 factory.go:223] Registration of the crio container factory successfully Jun 09 07:58:22.932953 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932849 2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 09 07:58:22.932953 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932882 2573 factory.go:103] Registering Raw factory Jun 09 07:58:22.932953 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.932894 2573 manager.go:1196] Started watching for new ooms in manager Jun 09 07:58:22.934036 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.934020 2573 manager.go:349] Starting recovery of all containers Jun 09 07:58:22.936347 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.936320 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6pcg7" Jun 09 07:58:22.945457 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.945256 2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 09 07:58:22.945568 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.945389 2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-141-98.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 09 07:58:22.945568 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.945446 2573 manager.go:354] Recovery completed Jun 09 07:58:22.950628 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.950614 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:22.952409 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.952388 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:22.952500 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.952425 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:22.952500 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.952440 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:22.953016 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953002 2573 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 09 07:58:22.953016 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953013 2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 09 07:58:22.953134 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953030 2573 state_mem.go:36] "Initialized new in-memory state store" Jun 09 07:58:22.953134 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953063 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:22.953851 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953836 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:22.953901 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953879 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:22.953901 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.953893 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:22.954293 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.954282 2573 policy_none.go:49] "None policy: Start" Jun 09 07:58:22.954744 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.954734 2573 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 09 07:58:22.954774 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.954753 2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 09 07:58:22.955712 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.955641 2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-98.ec2.internal.18b75ab31b53482d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-98.ec2.internal,UID:ip-10-0-141-98.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-141-98.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-141-98.ec2.internal,},FirstTimestamp:2026-06-09 07:58:22.952409133 +0000 UTC m=+0.392873102,LastTimestamp:2026-06-09 07:58:22.952409133 +0000 UTC m=+0.392873102,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-98.ec2.internal,}" Jun 09 07:58:22.955842 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.955830 2573 policy_none.go:47] "Start" Jun 09 07:58:22.958032 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.958015 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-6pcg7" Jun 09 07:58:22.987122 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987106 2573 manager.go:337] "Starting Device Plugin manager" Jun 09 07:58:22.987226 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.987139 2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 09 07:58:22.987226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987150 2573 server.go:87] "Starting device plugin registration server" Jun 09 07:58:22.987454 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987438 2573 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 09 07:58:22.987526 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987459 2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 09 07:58:22.987590 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987579 2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 09 07:58:22.987658 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987646 2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 09 07:58:22.987703 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:22.987658 2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 09 07:58:22.988282 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.988260 2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 09 07:58:22.988385 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:22.988317 2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.003648 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.003610 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 09 07:58:23.004944 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.004921 2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 09 07:58:23.005044 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.004974 2573 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 09 07:58:23.005044 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.005010 2573 kubelet.go:2448] "Starting kubelet main sync loop" Jun 09 07:58:23.005147 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.005054 2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 09 07:58:23.008343 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.008324 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:23.088346 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.088326 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:23.089312 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.089289 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:23.089423 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.089326 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:23.089423 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.089337 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:23.089423 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.089363 2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.099292 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.099267 2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.099292 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.099296 2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-98.ec2.internal\": node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.105481 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.105418 2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal"] Jun 09 07:58:23.105593 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.105494 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:23.106337 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.106318 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:23.106434 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.106357 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:23.106434 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.106373 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:23.107452 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.107439 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:23.107577 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.107562 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.107623 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.107614 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:23.108167 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108144 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:23.108167 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108170 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:23.108314 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108183 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:23.108314 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108207 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:23.108314 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108227 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:23.108314 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.108237 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:23.109410 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.109393 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.109452 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.109432 2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 09 07:58:23.110360 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.110344 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientMemory" Jun 09 07:58:23.110442 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.110368 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasNoDiskPressure" Jun 09 07:58:23.110442 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.110381 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeHasSufficientPID" Jun 09 07:58:23.117463 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.117446 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.125478 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.125454 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-98.ec2.internal\" not found" node="ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.130062 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.130046 2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-98.ec2.internal\" not found" node="ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.133502 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.133485 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9efa5c0d29862f4f6670610d25ee3730-config\") pod \"kube-apiserver-proxy-ip-10-0-141-98.ec2.internal\" (UID: \"9efa5c0d29862f4f6670610d25ee3730\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.133575 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.133509 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.133575 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.133526 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.218481 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.218454 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.233749 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9efa5c0d29862f4f6670610d25ee3730-config\") pod \"kube-apiserver-proxy-ip-10-0-141-98.ec2.internal\" (UID: \"9efa5c0d29862f4f6670610d25ee3730\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.233825 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233752 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.233825 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.233825 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233812 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.233958 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233836 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/24863a1ea4571aefe156d4d15df42186-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal\" (UID: \"24863a1ea4571aefe156d4d15df42186\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.233958 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.233836 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/9efa5c0d29862f4f6670610d25ee3730-config\") pod \"kube-apiserver-proxy-ip-10-0-141-98.ec2.internal\" (UID: \"9efa5c0d29862f4f6670610d25ee3730\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.319395 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.319365 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.419986 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.419906 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.429102 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.429072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.433559 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.433539 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.520382 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.520341 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.620854 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.620809 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.721401 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.721327 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.821895 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.821855 2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-98.ec2.internal\" not found" Jun 09 07:58:23.854947 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.854919 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:23.869734 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.869705 2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 09 07:58:23.869935 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.869909 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 07:58:23.869982 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.869910 2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 09 07:58:23.911048 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.911012 2573 apiserver.go:52] "Watching apiserver" Jun 09 07:58:23.925367 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:23.925337 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24863a1ea4571aefe156d4d15df42186.slice/crio-9d395f012ee16a98e3ade60dda0f15fcafded0a571bb6ccb940f89bbc9ee3647 WatchSource:0}: Error finding container 9d395f012ee16a98e3ade60dda0f15fcafded0a571bb6ccb940f89bbc9ee3647: Status 404 returned error can't find the container with id 9d395f012ee16a98e3ade60dda0f15fcafded0a571bb6ccb940f89bbc9ee3647 Jun 09 07:58:23.925752 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:23.925730 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efa5c0d29862f4f6670610d25ee3730.slice/crio-4641ae0aa48d9775647e1681e7482c9f5b1c71badde38b1de1c6f45840f4dd3f WatchSource:0}: Error finding container 4641ae0aa48d9775647e1681e7482c9f5b1c71badde38b1de1c6f45840f4dd3f: Status 404 returned error can't find the container with id 4641ae0aa48d9775647e1681e7482c9f5b1c71badde38b1de1c6f45840f4dd3f Jun 09 07:58:23.926254 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.926234 2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 09 07:58:23.926602 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.926581 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-p89d8","openshift-multus/multus-additional-cni-plugins-wkqzz","openshift-multus/network-metrics-daemon-h55kx","openshift-network-operator/iptables-alerter-dkwxm","openshift-ovn-kubernetes/ovnkube-node-ddz24","kube-system/global-pull-secret-syncer-glwhj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf","openshift-dns/node-resolver-jc2tq","openshift-multus/multus-mn8fx","openshift-network-diagnostics/network-check-target-pbs9g","kube-system/konnectivity-agent-5tn2p","openshift-cluster-node-tuning-operator/tuned-q52xc"] Jun 09 07:58:23.928520 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.928498 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:23.931305 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.930404 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.932666 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.932075 2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 09 07:58:23.932666 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.932372 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jvcf2\"" Jun 09 07:58:23.932666 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.932662 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.932892 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.932826 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.932892 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.932885 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 09 07:58:23.933170 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.933150 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.933546 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.933528 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.933674 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.933656 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-xtmf5\"" Jun 09 07:58:23.933806 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.933560 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 09 07:58:23.934067 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.934048 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:23.934067 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.934059 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 09 07:58:23.934207 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.934159 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:23.934294 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.934276 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 09 07:58:23.934703 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.933621 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.935162 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.935143 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 07:58:23.935377 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.935361 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:23.936388 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.936361 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:23.937069 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937049 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.937174 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937164 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937337 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-b4756\"" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937406 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937437 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm6r4\" (UniqueName: \"kubernetes.io/projected/8dc53d82-c5ab-4356-ad54-b17575987bc3-kube-api-access-xm6r4\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937467 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-host\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937490 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 09 07:58:23.937554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937490 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-serviceca\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937576 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937638 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937668 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jlvw\" (UniqueName: \"kubernetes.io/projected/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-kube-api-access-4jlvw\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937728 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2w4d\" (UniqueName: \"kubernetes.io/projected/e1e16526-f4bc-4e20-abb0-999881c8dcc4-kube-api-access-g2w4d\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937751 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-cnibin\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937771 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937775 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.937810 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-system-cni-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.937920 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.937852 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:23.938412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938027 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-os-release\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.938412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938054 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:23.938412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938148 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.938412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938377 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-84pnx\"" Jun 09 07:58:23.938412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938393 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 09 07:58:23.938688 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.938481 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 09 07:58:23.939147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.939127 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.939147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.939143 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 09 07:58:23.939510 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.939494 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 09 07:58:23.941036 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.941019 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:23.942216 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.942198 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:23.942715 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.942698 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 09 07:58:23.942786 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.942772 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mst5r\"" Jun 09 07:58:23.942825 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.942807 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.943070 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.943055 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.943720 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.943705 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mn8fx" Jun 09 07:58:23.945088 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.945072 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:23.945170 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:23.945117 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:23.946272 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.946254 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:23.947533 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.947509 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:23.949482 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949460 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.949573 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949504 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 07:58:23.949629 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949574 2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" Jun 09 07:58:23.949629 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949616 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal"] Jun 09 07:58:23.949729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949620 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 09 07:58:23.949729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.949684 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.950299 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.950282 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 09 07:58:23.950398 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.950355 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zd427\"" Jun 09 07:58:23.950456 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.950355 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-h6xx9\"" Jun 09 07:58:23.951032 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.951014 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 09 07:58:23.951226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.951207 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 09 07:58:23.951302 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.951222 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-tzhl9\"" Jun 09 07:58:23.951476 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.951461 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-clhnt\"" Jun 09 07:58:23.951741 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.951721 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 09 07:58:23.957701 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.957679 2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 09 07:58:23.960308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.960280 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-08 07:53:22 +0000 UTC" deadline="2027-12-08 16:41:22.403017826 +0000 UTC" Jun 09 07:58:23.960365 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.960311 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13136h42m58.442711208s" Jun 09 07:58:23.967709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.967343 2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 09 07:58:23.967805 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:23.967770 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal"] Jun 09 07:58:24.009451 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.009393 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" event={"ID":"9efa5c0d29862f4f6670610d25ee3730","Type":"ContainerStarted","Data":"4641ae0aa48d9775647e1681e7482c9f5b1c71badde38b1de1c6f45840f4dd3f"} Jun 09 07:58:24.010410 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.010386 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" event={"ID":"24863a1ea4571aefe156d4d15df42186","Type":"ContainerStarted","Data":"9d395f012ee16a98e3ade60dda0f15fcafded0a571bb6ccb940f89bbc9ee3647"} Jun 09 07:58:24.033698 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.033672 2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 09 07:58:24.037766 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.037750 2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cd9cd" Jun 09 07:58:24.038848 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.038833 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.038906 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.038858 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-kubelet\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.038906 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.038887 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-var-lib-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.038906 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.038903 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovn-node-metrics-cert\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.039040 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.038920 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-multus-certs\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.039104 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039079 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-sys\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039150 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039119 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-os-release\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.039150 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039143 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-bin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.039245 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039165 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-hostroot\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.039245 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039189 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-dbus\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.039245 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039213 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-lib-modules\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039245 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039239 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-serviceca\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039211 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-os-release\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039257 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039273 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-conf-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039309 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-sys-fs\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039327 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-run\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039347 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jlvw\" (UniqueName: \"kubernetes.io/projected/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-kube-api-access-4jlvw\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.039435 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039373 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-slash\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039469 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wmsw\" (UniqueName: \"kubernetes.io/projected/ed44772a-57dc-425a-8834-266066f4a961-kube-api-access-2wmsw\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039489 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rw8j2\" (UniqueName: \"kubernetes.io/projected/91a9c666-da0c-44de-97f3-e1d8d2d075cc-kube-api-access-rw8j2\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039537 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-node-log\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039554 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fxdj4\" (UniqueName: \"kubernetes.io/projected/e3c37a40-2c87-435e-a5e8-6fce48f481c9-kube-api-access-fxdj4\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039581 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysconfig\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039586 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-binary-copy\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039604 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-var-lib-kubelet\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039630 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-system-cni-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039666 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.039724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039666 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-serviceca\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039731 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-system-cni-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039776 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039802 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039815 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-conf\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039848 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-host\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039892 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-script-lib\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-host\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039917 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-system-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-cni-binary-copy\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-netns\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.039994 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040024 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040048 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-netns\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040066 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-systemd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040081 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040106 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040217 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040130 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-multus\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040145 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-etc-kubernetes\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040178 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040198 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-os-release\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040221 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrbdf\" (UniqueName: \"kubernetes.io/projected/7b762bfc-46af-4955-a39b-6e4f4369e856-kube-api-access-rrbdf\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040239 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dce1b52b-882e-4217-8fda-5317c83d4712-konnectivity-ca\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040268 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-systemd\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040286 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7p88\" (UniqueName: \"kubernetes.io/projected/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-kube-api-access-m7p88\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040308 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm6r4\" (UniqueName: \"kubernetes.io/projected/8dc53d82-c5ab-4356-ad54-b17575987bc3-kube-api-access-xm6r4\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040329 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-kubernetes\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040344 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-host\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040380 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-tuned\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-iptables-alerter-script\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040451 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040492 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed44772a-57dc-425a-8834-266066f4a961-tmp-dir\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.040796 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-daemon-config\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040601 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-socket-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040632 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-registration-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040657 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-bin\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040701 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-k8s-cni-cncf-io\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040747 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-netd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040770 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-config\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040793 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-env-overrides\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040817 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-device-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040843 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-tmp\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040889 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-etc-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040944 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-ovn\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040954 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040966 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed44772a-57dc-425a-8834-266066f4a961-hosts-file\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.041523 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.040989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-kubelet-config\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041011 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dce1b52b-882e-4217-8fda-5317c83d4712-agent-certs\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041045 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-modprobe-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041078 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041104 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-cnibin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041127 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-socket-dir-parent\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041152 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk94n\" (UniqueName: \"kubernetes.io/projected/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kube-api-access-pk94n\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.041170 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041175 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-host-slash\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041201 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2w4d\" (UniqueName: \"kubernetes.io/projected/e1e16526-f4bc-4e20-abb0-999881c8dcc4-kube-api-access-g2w4d\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.041250 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:24.541218003 +0000 UTC m=+1.981681962 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041263 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-systemd-units\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-log-socket\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041314 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-kubelet\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041323 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8dc53d82-c5ab-4356-ad54-b17575987bc3-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041340 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-cnibin\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.041985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.041422 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8dc53d82-c5ab-4356-ad54-b17575987bc3-cnibin\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.047334 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.047315 2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cd9cd" Jun 09 07:58:24.050532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.050509 2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 09 07:58:24.053723 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.053695 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jlvw\" (UniqueName: \"kubernetes.io/projected/2c562ebb-1fe5-491b-b3de-e5926b9d4cee-kube-api-access-4jlvw\") pod \"node-ca-p89d8\" (UID: \"2c562ebb-1fe5-491b-b3de-e5926b9d4cee\") " pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.053815 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.053699 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm6r4\" (UniqueName: \"kubernetes.io/projected/8dc53d82-c5ab-4356-ad54-b17575987bc3-kube-api-access-xm6r4\") pod \"multus-additional-cni-plugins-wkqzz\" (UID: \"8dc53d82-c5ab-4356-ad54-b17575987bc3\") " pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.053815 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.053768 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2w4d\" (UniqueName: \"kubernetes.io/projected/e1e16526-f4bc-4e20-abb0-999881c8dcc4-kube-api-access-g2w4d\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:24.141765 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141714 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rw8j2\" (UniqueName: \"kubernetes.io/projected/91a9c666-da0c-44de-97f3-e1d8d2d075cc-kube-api-access-rw8j2\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.141765 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-node-log\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141785 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fxdj4\" (UniqueName: \"kubernetes.io/projected/e3c37a40-2c87-435e-a5e8-6fce48f481c9-kube-api-access-fxdj4\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141836 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysconfig\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-node-log\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-var-lib-kubelet\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141925 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysconfig\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141949 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-conf\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.141989 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142000 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-script-lib\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142022 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142023 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-system-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142019 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-var-lib-kubelet\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142044 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-cni-binary-copy\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142067 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-netns\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142096 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-system-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142134 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142148 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-netns\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142154 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-conf\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142165 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142192 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-netns\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142216 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-systemd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.142221 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142241 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142279 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-systemd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142241 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142280 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142271 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-run-netns\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.142445 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.142272 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:24.642252011 +0000 UTC m=+2.082715972 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142366 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142395 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-multus\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142430 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-etc-kubernetes\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142454 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-multus\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142455 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142491 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-cni-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-os-release\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142584 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrbdf\" (UniqueName: \"kubernetes.io/projected/7b762bfc-46af-4955-a39b-6e4f4369e856-kube-api-access-rrbdf\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142593 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-script-lib\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142610 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dce1b52b-882e-4217-8fda-5317c83d4712-konnectivity-ca\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142555 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-etc-selinux\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142611 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-os-release\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142642 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-etc-kubernetes\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142700 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-cni-binary-copy\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142730 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-systemd\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7p88\" (UniqueName: \"kubernetes.io/projected/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-kube-api-access-m7p88\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.143226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142821 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-systemd\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142844 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-kubernetes\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142886 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-host\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142911 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-tuned\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142935 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-iptables-alerter-script\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142962 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed44772a-57dc-425a-8834-266066f4a961-tmp-dir\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142988 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-daemon-config\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.142986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-host\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143006 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-kubernetes\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143058 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-socket-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143089 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-registration-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143122 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-bin\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143146 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-k8s-cni-cncf-io\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/dce1b52b-882e-4217-8fda-5317c83d4712-konnectivity-ca\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143168 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143194 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-socket-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143199 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-netd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144091 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143234 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-bin\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143236 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-config\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143270 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-k8s-cni-cncf-io\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143238 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-cni-netd\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143272 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-env-overrides\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143316 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed44772a-57dc-425a-8834-266066f4a961-tmp-dir\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143325 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-registration-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143341 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-device-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143365 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-tmp\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143388 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-sysctl-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143402 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-etc-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143425 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-ovn\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143450 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed44772a-57dc-425a-8834-266066f4a961-hosts-file\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143474 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-kubelet-config\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143497 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dce1b52b-882e-4217-8fda-5317c83d4712-agent-certs\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143521 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-modprobe-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143527 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-etc-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143535 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-daemon-config\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.144941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143541 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-iptables-alerter-script\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143563 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-cnibin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143579 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-device-dir\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-socket-dir-parent\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143602 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ed44772a-57dc-425a-8834-266066f4a961-hosts-file\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143620 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk94n\" (UniqueName: \"kubernetes.io/projected/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kube-api-access-pk94n\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143626 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-kubelet-config\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143647 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-host-slash\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143653 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-cnibin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143673 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-systemd-units\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143704 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-log-socket\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-socket-dir-parent\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143498 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-ovn\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143710 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovnkube-config\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143735 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-kubelet\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143765 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-host-slash\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143768 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-kubelet\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143795 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-var-lib-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.145797 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143807 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-log-socket\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143810 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-kubelet\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovn-node-metrics-cert\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143883 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-kubelet\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143886 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-var-lib-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-multus-certs\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143770 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-systemd-units\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143925 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-sys\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143932 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-run-multus-certs\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143970 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-bin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.143996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-hostroot\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144004 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-sys\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144009 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e3c37a40-2c87-435e-a5e8-6fce48f481c9-env-overrides\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144022 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-dbus\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144045 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-lib-modules\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144063 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-hostroot\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-host-var-lib-cni-bin\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144074 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.146532 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144107 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-run-openvswitch\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144105 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-conf-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144141 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-sys-fs\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144146 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-lib-modules\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144141 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7b762bfc-46af-4955-a39b-6e4f4369e856-multus-conf-dir\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144147 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-dbus\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144191 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-run\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144191 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-sys-fs\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144219 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-slash\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144239 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-run\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144245 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-modprobe-d\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144248 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e3c37a40-2c87-435e-a5e8-6fce48f481c9-host-slash\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.144266 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wmsw\" (UniqueName: \"kubernetes.io/projected/ed44772a-57dc-425a-8834-266066f4a961-kube-api-access-2wmsw\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.145485 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-etc-tuned\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.146241 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/dce1b52b-882e-4217-8fda-5317c83d4712-agent-certs\") pod \"konnectivity-agent-5tn2p\" (UID: \"dce1b52b-882e-4217-8fda-5317c83d4712\") " pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.146372 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/91a9c666-da0c-44de-97f3-e1d8d2d075cc-tmp\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.147045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.146384 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e3c37a40-2c87-435e-a5e8-6fce48f481c9-ovn-node-metrics-cert\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.161745 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.161718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk94n\" (UniqueName: \"kubernetes.io/projected/e5160723-c72a-43e9-8cb0-3f7c8848b2d9-kube-api-access-pk94n\") pod \"aws-ebs-csi-driver-node-xxmhf\" (UID: \"e5160723-c72a-43e9-8cb0-3f7c8848b2d9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.162759 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.162733 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrbdf\" (UniqueName: \"kubernetes.io/projected/7b762bfc-46af-4955-a39b-6e4f4369e856-kube-api-access-rrbdf\") pod \"multus-mn8fx\" (UID: \"7b762bfc-46af-4955-a39b-6e4f4369e856\") " pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.163351 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.163332 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:24.163351 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.163353 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:24.163501 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.163363 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:24.163501 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.163418 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:24.663402966 +0000 UTC m=+2.103866921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:24.163735 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.163718 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fxdj4\" (UniqueName: \"kubernetes.io/projected/e3c37a40-2c87-435e-a5e8-6fce48f481c9-kube-api-access-fxdj4\") pod \"ovnkube-node-ddz24\" (UID: \"e3c37a40-2c87-435e-a5e8-6fce48f481c9\") " pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.165800 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.165776 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wmsw\" (UniqueName: \"kubernetes.io/projected/ed44772a-57dc-425a-8834-266066f4a961-kube-api-access-2wmsw\") pod \"node-resolver-jc2tq\" (UID: \"ed44772a-57dc-425a-8834-266066f4a961\") " pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.166190 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.166172 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rw8j2\" (UniqueName: \"kubernetes.io/projected/91a9c666-da0c-44de-97f3-e1d8d2d075cc-kube-api-access-rw8j2\") pod \"tuned-q52xc\" (UID: \"91a9c666-da0c-44de-97f3-e1d8d2d075cc\") " pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.166339 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.166325 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7p88\" (UniqueName: \"kubernetes.io/projected/b77be634-4ae4-4004-a4c0-4e79b5fd7a0c-kube-api-access-m7p88\") pod \"iptables-alerter-dkwxm\" (UID: \"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c\") " pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.242290 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.242220 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-p89d8" Jun 09 07:58:24.249159 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.249134 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" Jun 09 07:58:24.249964 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.249944 2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:24.250798 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.250769 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c562ebb_1fe5_491b_b3de_e5926b9d4cee.slice/crio-44212ae14a6d5a6def5addfe621af6e732f3381d04df90dd618b264cda5fe804 WatchSource:0}: Error finding container 44212ae14a6d5a6def5addfe621af6e732f3381d04df90dd618b264cda5fe804: Status 404 returned error can't find the container with id 44212ae14a6d5a6def5addfe621af6e732f3381d04df90dd618b264cda5fe804 Jun 09 07:58:24.256062 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.256036 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc53d82_c5ab_4356_ad54_b17575987bc3.slice/crio-584c5f6ca7cfe604db699824956ebd20ec04a364f98b1537d5c5250026be535e WatchSource:0}: Error finding container 584c5f6ca7cfe604db699824956ebd20ec04a364f98b1537d5c5250026be535e: Status 404 returned error can't find the container with id 584c5f6ca7cfe604db699824956ebd20ec04a364f98b1537d5c5250026be535e Jun 09 07:58:24.257042 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.257026 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dkwxm" Jun 09 07:58:24.262012 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.261992 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:24.263949 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.263924 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb77be634_4ae4_4004_a4c0_4e79b5fd7a0c.slice/crio-ab262d2cb43aca1fdf34cc1a7824f0e7b8eb200a0bb1b5c5a3fe4f5f0da4adcd WatchSource:0}: Error finding container ab262d2cb43aca1fdf34cc1a7824f0e7b8eb200a0bb1b5c5a3fe4f5f0da4adcd: Status 404 returned error can't find the container with id ab262d2cb43aca1fdf34cc1a7824f0e7b8eb200a0bb1b5c5a3fe4f5f0da4adcd Jun 09 07:58:24.267929 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.267908 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" Jun 09 07:58:24.268920 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.268899 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c37a40_2c87_435e_a5e8_6fce48f481c9.slice/crio-10d7af55b104e10769b534aea914a9cc0bf816c475e36159df410f0846a32fb9 WatchSource:0}: Error finding container 10d7af55b104e10769b534aea914a9cc0bf816c475e36159df410f0846a32fb9: Status 404 returned error can't find the container with id 10d7af55b104e10769b534aea914a9cc0bf816c475e36159df410f0846a32fb9 Jun 09 07:58:24.275645 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.275625 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jc2tq" Jun 09 07:58:24.275907 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.275887 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5160723_c72a_43e9_8cb0_3f7c8848b2d9.slice/crio-6eab8ae5197786a7ab59805374f5ed026fec62e6327c089d27e9b0fa028e25b1 WatchSource:0}: Error finding container 6eab8ae5197786a7ab59805374f5ed026fec62e6327c089d27e9b0fa028e25b1: Status 404 returned error can't find the container with id 6eab8ae5197786a7ab59805374f5ed026fec62e6327c089d27e9b0fa028e25b1 Jun 09 07:58:24.280448 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.280428 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mn8fx" Jun 09 07:58:24.282423 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.282405 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded44772a_57dc_425a_8834_266066f4a961.slice/crio-7ebc78e74b7f9539464ec34d0dcbcd1ef8658e13fb230d2364df790b1ad4f546 WatchSource:0}: Error finding container 7ebc78e74b7f9539464ec34d0dcbcd1ef8658e13fb230d2364df790b1ad4f546: Status 404 returned error can't find the container with id 7ebc78e74b7f9539464ec34d0dcbcd1ef8658e13fb230d2364df790b1ad4f546 Jun 09 07:58:24.285726 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.285701 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:24.290498 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.290473 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-q52xc" Jun 09 07:58:24.291348 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.291324 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b762bfc_46af_4955_a39b_6e4f4369e856.slice/crio-3e5764465ac4e1a3a66fce3137d6faada7f9b32b06e7d04c808eabf20ef4e744 WatchSource:0}: Error finding container 3e5764465ac4e1a3a66fce3137d6faada7f9b32b06e7d04c808eabf20ef4e744: Status 404 returned error can't find the container with id 3e5764465ac4e1a3a66fce3137d6faada7f9b32b06e7d04c808eabf20ef4e744 Jun 09 07:58:24.294933 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.294826 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddce1b52b_882e_4217_8fda_5317c83d4712.slice/crio-a54a65d5c07b3cb261c6a386e6a162846003da741e0e612d1f1877de437ca1b8 WatchSource:0}: Error finding container a54a65d5c07b3cb261c6a386e6a162846003da741e0e612d1f1877de437ca1b8: Status 404 returned error can't find the container with id a54a65d5c07b3cb261c6a386e6a162846003da741e0e612d1f1877de437ca1b8 Jun 09 07:58:24.300170 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:24.300145 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91a9c666_da0c_44de_97f3_e1d8d2d075cc.slice/crio-3816e157a8a637594e24f06400765a19997b5a103af68b1ca21843864606fbc2 WatchSource:0}: Error finding container 3816e157a8a637594e24f06400765a19997b5a103af68b1ca21843864606fbc2: Status 404 returned error can't find the container with id 3816e157a8a637594e24f06400765a19997b5a103af68b1ca21843864606fbc2 Jun 09 07:58:24.364457 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.364412 2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:24.547011 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.546931 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:24.547173 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.547133 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:24.547291 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.547205 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:25.547185637 +0000 UTC m=+2.987649596 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:24.647714 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.647679 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:24.647952 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.647934 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:24.648044 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.648019 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:25.647998543 +0000 UTC m=+3.088462506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:24.726501 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.726467 2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:24.749306 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:24.748645 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:24.749306 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.748841 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:24.749306 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.748883 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:24.749306 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.748896 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:24.749306 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:24.748954 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:25.748932777 +0000 UTC m=+3.189396745 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:25.032707 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.032605 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mn8fx" event={"ID":"7b762bfc-46af-4955-a39b-6e4f4369e856","Type":"ContainerStarted","Data":"3e5764465ac4e1a3a66fce3137d6faada7f9b32b06e7d04c808eabf20ef4e744"} Jun 09 07:58:25.050209 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.050143 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 07:53:24 +0000 UTC" deadline="2028-02-23 02:18:27.236624852 +0000 UTC" Jun 09 07:58:25.050209 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.050207 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14970h20m2.18642128s" Jun 09 07:58:25.078367 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.078325 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jc2tq" event={"ID":"ed44772a-57dc-425a-8834-266066f4a961","Type":"ContainerStarted","Data":"7ebc78e74b7f9539464ec34d0dcbcd1ef8658e13fb230d2364df790b1ad4f546"} Jun 09 07:58:25.097277 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.097151 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"10d7af55b104e10769b534aea914a9cc0bf816c475e36159df410f0846a32fb9"} Jun 09 07:58:25.122433 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.122291 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q52xc" event={"ID":"91a9c666-da0c-44de-97f3-e1d8d2d075cc","Type":"ContainerStarted","Data":"3816e157a8a637594e24f06400765a19997b5a103af68b1ca21843864606fbc2"} Jun 09 07:58:25.138626 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.136059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5tn2p" event={"ID":"dce1b52b-882e-4217-8fda-5317c83d4712","Type":"ContainerStarted","Data":"a54a65d5c07b3cb261c6a386e6a162846003da741e0e612d1f1877de437ca1b8"} Jun 09 07:58:25.142263 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.142141 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" event={"ID":"e5160723-c72a-43e9-8cb0-3f7c8848b2d9","Type":"ContainerStarted","Data":"6eab8ae5197786a7ab59805374f5ed026fec62e6327c089d27e9b0fa028e25b1"} Jun 09 07:58:25.164547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.164379 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dkwxm" event={"ID":"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c","Type":"ContainerStarted","Data":"ab262d2cb43aca1fdf34cc1a7824f0e7b8eb200a0bb1b5c5a3fe4f5f0da4adcd"} Jun 09 07:58:25.173881 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.173811 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerStarted","Data":"584c5f6ca7cfe604db699824956ebd20ec04a364f98b1537d5c5250026be535e"} Jun 09 07:58:25.179098 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.179048 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p89d8" event={"ID":"2c562ebb-1fe5-491b-b3de-e5926b9d4cee","Type":"ContainerStarted","Data":"44212ae14a6d5a6def5addfe621af6e732f3381d04df90dd618b264cda5fe804"} Jun 09 07:58:25.553919 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.553877 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:25.554161 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.554143 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:25.554231 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.554217 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:27.554196393 +0000 UTC m=+4.994660364 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:25.654942 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.654893 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:25.655129 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.655117 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:25.655194 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.655180 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:27.655161766 +0000 UTC m=+5.095625727 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:25.756399 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:25.755767 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:25.756399 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.755964 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:25.756399 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.755986 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:25.756399 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.756001 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:25.756399 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:25.756063 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:27.756043892 +0000 UTC m=+5.196507872 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:26.005761 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.005603 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:26.005761 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:26.005736 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:26.006184 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.006163 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:26.006299 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:26.006279 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:26.006360 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.006349 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:26.006437 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:26.006419 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:26.051205 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.051163 2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-08 07:53:24 +0000 UTC" deadline="2027-11-28 12:52:29.283222733 +0000 UTC" Jun 09 07:58:26.051205 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.051199 2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12892h54m3.232029538s" Jun 09 07:58:26.227111 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:26.227075 2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 09 07:58:27.573113 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:27.573073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:27.573566 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.573229 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:27.573566 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.573308 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:31.573289595 +0000 UTC m=+9.013753565 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:27.674239 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:27.674200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:27.674421 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.674347 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:27.674421 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.674410 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:31.674392969 +0000 UTC m=+9.114856931 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:27.775680 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:27.774985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:27.775680 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.775193 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:27.775680 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.775216 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:27.775680 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.775232 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:27.775680 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:27.775293 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:31.775274318 +0000 UTC m=+9.215738297 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:28.005750 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:28.005645 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:28.005928 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:28.005787 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:28.006018 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:28.005945 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:28.006088 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:28.006064 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:28.006178 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:28.006133 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:28.006386 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:28.006225 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:30.005780 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:30.005741 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:30.006239 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:30.005741 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:30.006239 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:30.005887 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:30.006239 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:30.006020 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:30.006239 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:30.005748 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:30.006239 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:30.006134 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:31.609990 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:31.609883 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:31.610453 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.610007 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:31.610453 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.610073 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:39.610055744 +0000 UTC m=+17.050519728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:31.711074 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:31.711031 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:31.711281 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.711209 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:31.711347 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.711300 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:39.711277927 +0000 UTC m=+17.151741885 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:31.813190 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:31.812200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:31.813190 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.812449 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:31.813190 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.812467 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:31.813190 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.812518 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:31.813190 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:31.812586 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:39.812566428 +0000 UTC m=+17.253030395 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:32.005729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:32.005490 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:32.005729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:32.005514 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:32.005729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:32.005495 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:32.005729 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:32.005643 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:32.006084 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:32.005739 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:32.006084 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:32.005795 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:34.006394 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:34.006304 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:34.006829 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:34.006310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:34.006829 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:34.006471 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:34.006829 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:34.006310 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:34.006829 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:34.006553 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:34.006829 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:34.006619 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:36.005963 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:36.005693 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:36.006368 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:36.005711 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:36.006368 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:36.006064 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:36.006368 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:36.006181 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:36.006368 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:36.005723 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:36.006368 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:36.006293 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:38.005425 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:38.005390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:38.005853 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:38.005390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:38.005853 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:38.005390 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:38.005853 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:38.005563 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:38.005853 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:38.005644 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:38.005853 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:38.005741 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:39.670886 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:39.670842 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:39.671262 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.670972 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:39.671262 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.671038 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:55.671021884 +0000 UTC m=+33.111485859 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:39.771319 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:39.771202 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:39.771485 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.771369 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:39.771485 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.771430 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:55.771410922 +0000 UTC m=+33.211874878 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:39.871675 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:39.871637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:39.871901 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.871851 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:39.871901 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.871892 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:39.872014 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.871908 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:39.872014 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:39.871969 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:58:55.871953852 +0000 UTC m=+33.312417812 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:40.005763 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:40.005719 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:40.005763 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:40.005755 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:40.006032 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:40.005880 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:40.006032 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:40.005882 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:40.006032 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:40.005990 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:40.006205 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:40.006086 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:42.006203 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:42.006167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:42.006203 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:42.006194 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:42.006803 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:42.006167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:42.006803 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:42.006317 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:42.006803 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:42.006434 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:42.006803 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:42.006506 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:44.005404 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:44.005371 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:44.005404 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:44.005392 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:44.005925 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:44.005395 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:44.005925 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:44.005483 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:44.005925 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:44.005585 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:44.005925 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:44.005689 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:45.227568 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.227367 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jc2tq" event={"ID":"ed44772a-57dc-425a-8834-266066f4a961","Type":"ContainerStarted","Data":"94d2ca11818077f4a8eac6b801dc7fa22c24077c2a25132f2bb0a3669940a452"} Jun 09 07:58:45.230928 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.230898 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"54610517d3e4cc0aeacb79d9dce99ad364fc9f13c7cae6e834984011a6a40244"} Jun 09 07:58:45.231100 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.230986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"e1bee023bd8ade74a8303dd6cfd179bc5d981a703df77ee77759ce86e31c572b"} Jun 09 07:58:45.231100 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.231037 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"1c22fdfb50f629dab58c586682b6a19e1db858661f0d24ac8b524685d96e1a4a"} Jun 09 07:58:45.231100 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.231083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"a7dee816eec703b12a01e2fa99c0227b4054c7fb18a742fbf5fa1466f5597065"} Jun 09 07:58:45.231264 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.231129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"e99182f5a524560be96363bf1bb6812b372197b8bed093c234081a863fba71cb"} Jun 09 07:58:45.231264 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.231187 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"4d8a8afb31c82ebf3cd86eba565fbbc3ca4091fd9834481248cc740bed84539f"} Jun 09 07:58:45.232649 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.232610 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" event={"ID":"9efa5c0d29862f4f6670610d25ee3730","Type":"ContainerStarted","Data":"59c41fc21d9782faa2ca7ddc7cc47ed64906ef28419e1297b323196e0b83d912"} Jun 09 07:58:45.234184 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.234154 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-q52xc" event={"ID":"91a9c666-da0c-44de-97f3-e1d8d2d075cc","Type":"ContainerStarted","Data":"e59a56abba5aff395f992f6a83e60cc54d5ee9ebd38096a3c45c637bc36603fb"} Jun 09 07:58:45.235580 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.235553 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5tn2p" event={"ID":"dce1b52b-882e-4217-8fda-5317c83d4712","Type":"ContainerStarted","Data":"c5b36295701fc6d3989a764782fd798abf4002b951b3ffcaebc53484c50c1971"} Jun 09 07:58:45.237194 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.237171 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" event={"ID":"e5160723-c72a-43e9-8cb0-3f7c8848b2d9","Type":"ContainerStarted","Data":"292df90c5c51e7a32827331fd634d459bcdd6b6484bb49714d6d5a373f1458f3"} Jun 09 07:58:45.238566 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.238537 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="0922fbc95e9286639705ecb92a96bdfe37b4c8abb2428a419887ad038ce1dbc5" exitCode=0 Jun 09 07:58:45.238681 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.238578 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"0922fbc95e9286639705ecb92a96bdfe37b4c8abb2428a419887ad038ce1dbc5"} Jun 09 07:58:45.240169 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.240132 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-p89d8" event={"ID":"2c562ebb-1fe5-491b-b3de-e5926b9d4cee","Type":"ContainerStarted","Data":"8e64dfa343f8e4b4d7fc43cb22f8a0a346db0b7ad1fbd83f6e8dab9f1b207fa8"} Jun 09 07:58:45.241634 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.241613 2573 generic.go:356] "Generic (PLEG): container finished" podID="24863a1ea4571aefe156d4d15df42186" containerID="b7a9ac58551ba939d45d28ee0a730ae84f3dffdd83344543943678f619dfd9cb" exitCode=0 Jun 09 07:58:45.241705 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.241678 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" event={"ID":"24863a1ea4571aefe156d4d15df42186","Type":"ContainerDied","Data":"b7a9ac58551ba939d45d28ee0a730ae84f3dffdd83344543943678f619dfd9cb"} Jun 09 07:58:45.242933 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.242914 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mn8fx" event={"ID":"7b762bfc-46af-4955-a39b-6e4f4369e856","Type":"ContainerStarted","Data":"8a1dedd7b964b2fb61581d9fbbdf52ccfa90a3f637afcb0b46e97f70d072e37c"} Jun 09 07:58:45.244554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.244513 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jc2tq" podStartSLOduration=2.228340252 podStartE2EDuration="22.244502045s" totalImagesPullingTime="20.016161793s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.244059363 +0000 UTC m=+22.684523341" watchObservedRunningTime="2026-06-09 07:58:45.244502045 +0000 UTC m=+22.684966023" Jun 09 07:58:45.258763 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.258718 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-q52xc" podStartSLOduration=2.258571102 podStartE2EDuration="22.258706094s" totalImagesPullingTime="20.000134992s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.25852258 +0000 UTC m=+22.698986558" watchObservedRunningTime="2026-06-09 07:58:45.258706094 +0000 UTC m=+22.699170073" Jun 09 07:58:45.270249 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.270202 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-98.ec2.internal" podStartSLOduration=22.270188092 podStartE2EDuration="22.270188092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.269811346 +0000 UTC m=+22.710275325" watchObservedRunningTime="2026-06-09 07:58:45.270188092 +0000 UTC m=+22.710652157" Jun 09 07:58:45.284102 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.284057 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-p89d8" podStartSLOduration=2.269230076 podStartE2EDuration="22.284023962s" totalImagesPullingTime="20.014793886s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.283589001 +0000 UTC m=+22.724052983" watchObservedRunningTime="2026-06-09 07:58:45.284023962 +0000 UTC m=+22.724487996" Jun 09 07:58:45.300342 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.300230 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mn8fx" podStartSLOduration=2.288406964 podStartE2EDuration="22.300211985s" totalImagesPullingTime="20.011805021s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.29952457 +0000 UTC m=+22.739988548" watchObservedRunningTime="2026-06-09 07:58:45.300211985 +0000 UTC m=+22.740675962" Jun 09 07:58:45.316337 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.316285 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5tn2p" podStartSLOduration=2.34558426 podStartE2EDuration="22.316271024s" totalImagesPullingTime="19.970686764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:45.315817932 +0000 UTC m=+22.756281909" watchObservedRunningTime="2026-06-09 07:58:45.316271024 +0000 UTC m=+22.756735002" Jun 09 07:58:45.364169 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.364128 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:45.428626 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.428597 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:45.499736 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:45.499702 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:46.006197 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.006108 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:46.006197 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.006136 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:46.006492 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.006116 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:46.006492 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:46.006273 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:46.006492 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:46.006358 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:46.006492 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:46.006424 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:46.247452 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.247411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dkwxm" event={"ID":"b77be634-4ae4-4004-a4c0-4e79b5fd7a0c","Type":"ContainerStarted","Data":"3d2a7c7633b8c3f3b2f4e06a5e0133f8e4aaa9626e758022a25efef5d28340c6"} Jun 09 07:58:46.277517 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.277412 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dkwxm" podStartSLOduration=3.276225109 podStartE2EDuration="23.277394996s" totalImagesPullingTime="20.001169887s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:46.261910831 +0000 UTC m=+23.702374820" watchObservedRunningTime="2026-06-09 07:58:46.277394996 +0000 UTC m=+23.717858974" Jun 09 07:58:46.313349 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.313312 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:46.314604 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.314577 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:46.363895 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.363846 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:46.464885 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.464795 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:46.508967 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:46.508936 2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 09 07:58:47.001135 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.001004 2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-09T07:58:46.508955925Z","UUID":"9c1fe25b-aa77-45b2-860c-216a02186bcf","Handler":null,"Name":"","Endpoint":""} Jun 09 07:58:47.003926 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.003896 2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 09 07:58:47.004092 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.003945 2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 09 07:58:47.252941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.252842 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" event={"ID":"e5160723-c72a-43e9-8cb0-3f7c8848b2d9","Type":"ContainerStarted","Data":"aed8fcf84de02a0fd7232621f2de62144dd66d1440cc0fde492cfde69d56a200"} Jun 09 07:58:47.256066 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.255936 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" event={"ID":"24863a1ea4571aefe156d4d15df42186","Type":"ContainerStarted","Data":"4649f5bf14e1a0de9c50f69d5d33cc9117dd1b5c9f715e21c8b90cfe20056b30"} Jun 09 07:58:47.257475 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.256528 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:47.257475 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.257134 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5tn2p" Jun 09 07:58:47.274113 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.274059 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-98.ec2.internal" podStartSLOduration=24.274036435 podStartE2EDuration="24.274036435s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:47.273113601 +0000 UTC m=+24.713577580" watchObservedRunningTime="2026-06-09 07:58:47.274036435 +0000 UTC m=+24.714500416" Jun 09 07:58:47.378726 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:47.378687 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:48.005359 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:48.005320 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:48.005359 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:48.005341 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:48.005722 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:48.005326 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:48.005722 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:48.005454 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:48.005722 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:48.005545 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:48.005722 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:48.005643 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:48.262334 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:48.262248 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"1e9ae55f018c84fe31a3e0749c486ce402844ab4cb102bb2449e443da0da2cff"} Jun 09 07:58:50.006336 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.006164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:50.007003 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.006164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:50.007003 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:50.006423 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:50.007003 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.006164 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:50.007003 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:50.006485 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:50.007003 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:50.006553 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:50.269003 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.268925 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" event={"ID":"e5160723-c72a-43e9-8cb0-3f7c8848b2d9","Type":"ContainerStarted","Data":"315c1984cc8d27b0007f73538087e6f2e1489599bbfcf874661f3850a3dd075a"} Jun 09 07:58:50.270895 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.270636 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="ae5b655134ba2bd902220db98cdeb31eb5268755872933df25b1adc2737ebd09" exitCode=0 Jun 09 07:58:50.271034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.270692 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"ae5b655134ba2bd902220db98cdeb31eb5268755872933df25b1adc2737ebd09"} Jun 09 07:58:50.274710 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.274691 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" event={"ID":"e3c37a40-2c87-435e-a5e8-6fce48f481c9","Type":"ContainerStarted","Data":"9663747ce5eb7682bd2258041b39ea5454ce981a2958257ef5374c07ee5c686c"} Jun 09 07:58:50.274985 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.274965 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:50.275082 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.275040 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:50.287554 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.287507 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xxmhf" podStartSLOduration=3.257890205 podStartE2EDuration="27.287491137s" totalImagesPullingTime="24.029600932s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:50.287106572 +0000 UTC m=+27.727570553" watchObservedRunningTime="2026-06-09 07:58:50.287491137 +0000 UTC m=+27.727955116" Jun 09 07:58:50.291436 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.291412 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:50.295312 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.295292 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:50.311750 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:50.311709 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" podStartSLOduration=7.170883904 podStartE2EDuration="27.311694168s" totalImagesPullingTime="20.140810264s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:58:50.310730365 +0000 UTC m=+27.751194344" watchObservedRunningTime="2026-06-09 07:58:50.311694168 +0000 UTC m=+27.752158146" Jun 09 07:58:51.277527 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.277500 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 07:58:51.519634 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.519598 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h55kx"] Jun 09 07:58:51.519808 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.519759 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:51.520043 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:51.519890 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:51.522964 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.522932 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-glwhj"] Jun 09 07:58:51.523097 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.523051 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:51.523184 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:51.523161 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:51.523497 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.523478 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pbs9g"] Jun 09 07:58:51.523611 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:51.523581 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:51.523665 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:51.523654 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:52.198453 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:52.198419 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:52.255211 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:52.255177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:52.281676 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:52.281640 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="b494497501928c895a74232905f4f0f505050bc24ccf82d950cdfaa4a34aef15" exitCode=0 Jun 09 07:58:52.282099 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:52.281722 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"b494497501928c895a74232905f4f0f505050bc24ccf82d950cdfaa4a34aef15"} Jun 09 07:58:52.282099 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:52.281807 2573 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 09 07:58:53.006529 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:53.006499 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:53.006529 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:53.006577 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:53.006845 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:53.006655 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:53.006845 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:53.006741 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:53.006845 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:53.006782 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:53.006845 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:53.006839 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:53.286672 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:53.286633 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerStarted","Data":"8a611941ad64bdbcbaead437a10c70efe725107f63bf094c8e2fccaa7576d9cd"} Jun 09 07:58:53.867766 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:53.867532 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:58:54.291293 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:54.291259 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="8a611941ad64bdbcbaead437a10c70efe725107f63bf094c8e2fccaa7576d9cd" exitCode=0 Jun 09 07:58:54.291684 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:54.291304 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"8a611941ad64bdbcbaead437a10c70efe725107f63bf094c8e2fccaa7576d9cd"} Jun 09 07:58:55.005258 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.005226 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:55.005456 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.005226 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:55.005456 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.005365 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:55.005456 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.005429 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:55.005456 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.005226 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:55.005669 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.005522 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:55.690587 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.690547 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:55.691088 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.690682 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:55.691088 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.690755 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:27.690733153 +0000 UTC m=+65.131197114 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 09 07:58:55.791466 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.791433 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:55.791651 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.791605 2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:55.791708 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.791684 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret podName:9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:27.791662293 +0000 UTC m=+65.232126254 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret") pod "global-pull-secret-syncer-glwhj" (UID: "9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4") : object "kube-system"/"original-pull-secret" not registered Jun 09 07:58:55.892681 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:55.892637 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:55.892877 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.892836 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 09 07:58:55.892924 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.892883 2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 09 07:58:55.892924 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.892899 2573 projected.go:196] Error preparing data for projected volume kube-api-access-58zb8 for pod openshift-network-diagnostics/network-check-target-pbs9g: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:55.893013 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:55.892967 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8 podName:384ac8e6-a50f-400f-ae74-71bcd97bfdfd nodeName:}" failed. No retries permitted until 2026-06-09 07:59:27.892944623 +0000 UTC m=+65.333408602 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-58zb8" (UniqueName: "kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8") pod "network-check-target-pbs9g" (UID: "384ac8e6-a50f-400f-ae74-71bcd97bfdfd") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 09 07:58:57.005775 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:57.005734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:57.006307 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:57.005885 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pbs9g" podUID="384ac8e6-a50f-400f-ae74-71bcd97bfdfd" Jun 09 07:58:57.006307 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:57.005734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:57.006307 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:57.005981 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-glwhj" podUID="9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4" Jun 09 07:58:57.006307 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:57.005734 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:57.006307 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:57.006041 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 07:58:58.304690 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.304649 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerStarted","Data":"eae34d6004e890ade36978c4bac258dafa5aeeff87b7f5b15a3842f26ffbd467"} Jun 09 07:58:58.398609 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.398527 2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-98.ec2.internal" event="NodeReady" Jun 09 07:58:58.398751 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.398658 2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 09 07:58:58.431875 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.431831 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f55f497b5-4bpqs"] Jun 09 07:58:58.435113 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.435097 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.437051 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.437025 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 09 07:58:58.437248 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.437230 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 09 07:58:58.437331 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.437246 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xh9rz\"" Jun 09 07:58:58.437331 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.437307 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 09 07:58:58.442717 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.442696 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jh5j9"] Jun 09 07:58:58.443561 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.443537 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 09 07:58:58.446135 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.446116 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f55f497b5-4bpqs"] Jun 09 07:58:58.446235 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.446219 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.448492 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.448464 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 09 07:58:58.448659 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.448632 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 09 07:58:58.449380 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.449357 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-bzcsd\"" Jun 09 07:58:58.449644 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.449479 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5v8n5"] Jun 09 07:58:58.449644 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.449540 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 09 07:58:58.452803 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.452786 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.456085 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.455248 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 09 07:58:58.456085 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.455540 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 09 07:58:58.456085 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.455829 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9bx48\"" Jun 09 07:58:58.461436 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.461411 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jh5j9"] Jun 09 07:58:58.463563 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.463542 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5v8n5"] Jun 09 07:58:58.510372 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510325 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510372 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b228ca84-f54d-4819-8ab9-bf4b8983ac69-cert\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.510586 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510396 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7n2f4\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510586 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510450 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510586 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510516 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510586 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510569 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510589 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510606 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.510709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510622 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqnkq\" (UniqueName: \"kubernetes.io/projected/b228ca84-f54d-4819-8ab9-bf4b8983ac69-kube-api-access-nqnkq\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.510709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.510677 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.535728 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.535697 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:58.611758 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611718 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.611758 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611759 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b228ca84-f54d-4819-8ab9-bf4b8983ac69-cert\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611780 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8b2f40-3541-4e14-906b-440cb76c416d-config-volume\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611798 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7n2f4\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611816 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611849 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611929 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4zch\" (UniqueName: \"kubernetes.io/projected/cd8b2f40-3541-4e14-906b-440cb76c416d-kube-api-access-x4zch\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611961 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.611998 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612034 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612021 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612046 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8b2f40-3541-4e14-906b-440cb76c416d-tmp-dir\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612073 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqnkq\" (UniqueName: \"kubernetes.io/projected/b228ca84-f54d-4819-8ab9-bf4b8983ac69-kube-api-access-nqnkq\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612095 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:58.612128 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:58.612148 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612194 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612418 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:58.612221 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:58:59.112198851 +0000 UTC m=+36.552662822 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:58:58.612954 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612423 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.612954 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.612753 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.613190 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.613167 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.615964 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.615941 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.616062 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.616029 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.616125 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.616083 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b228ca84-f54d-4819-8ab9-bf4b8983ac69-cert\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.619685 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.619664 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqnkq\" (UniqueName: \"kubernetes.io/projected/b228ca84-f54d-4819-8ab9-bf4b8983ac69-kube-api-access-nqnkq\") pod \"ingress-canary-jh5j9\" (UID: \"b228ca84-f54d-4819-8ab9-bf4b8983ac69\") " pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.619785 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.619687 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.619785 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.619758 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7n2f4\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:58.637223 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.637181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:58:58.713314 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.713282 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4zch\" (UniqueName: \"kubernetes.io/projected/cd8b2f40-3541-4e14-906b-440cb76c416d-kube-api-access-x4zch\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.713438 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.713338 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8b2f40-3541-4e14-906b-440cb76c416d-tmp-dir\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.713547 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.713520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.713628 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.713613 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8b2f40-3541-4e14-906b-440cb76c416d-config-volume\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.713666 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:58.713645 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:58:58.713710 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.713677 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd8b2f40-3541-4e14-906b-440cb76c416d-tmp-dir\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.713741 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:58.713710 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:58:59.213695229 +0000 UTC m=+36.654159185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:58:58.714133 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.714114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cd8b2f40-3541-4e14-906b-440cb76c416d-config-volume\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.721154 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.721134 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4zch\" (UniqueName: \"kubernetes.io/projected/cd8b2f40-3541-4e14-906b-440cb76c416d-kube-api-access-x4zch\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:58.757369 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.757322 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jh5j9" Jun 09 07:58:58.891001 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:58.890967 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jh5j9"] Jun 09 07:58:58.895466 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:58:58.895436 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb228ca84_f54d_4819_8ab9_bf4b8983ac69.slice/crio-58dc6bef384b6291d7b25fe7389ea28b6fd3dd097f97aa07db43584386e17358 WatchSource:0}: Error finding container 58dc6bef384b6291d7b25fe7389ea28b6fd3dd097f97aa07db43584386e17358: Status 404 returned error can't find the container with id 58dc6bef384b6291d7b25fe7389ea28b6fd3dd097f97aa07db43584386e17358 Jun 09 07:58:59.005995 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.005896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:58:59.006174 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.005897 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:58:59.006174 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.005903 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:58:59.008496 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008466 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 07:58:59.008496 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008481 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 07:58:59.008496 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008472 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dscxw\"" Jun 09 07:58:59.008750 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008501 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7k9q\"" Jun 09 07:58:59.008750 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008741 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 07:58:59.008963 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.008948 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 07:58:59.117039 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.116996 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:58:59.117186 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:59.117152 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:58:59.117186 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:59.117173 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:58:59.117282 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:59.117230 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:00.117213508 +0000 UTC m=+37.557677466 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:58:59.217612 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.217575 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:58:59.217798 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:59.217705 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:58:59.217798 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:58:59.217769 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:59:00.217752121 +0000 UTC m=+37.658216100 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:58:59.307739 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.307653 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jh5j9" event={"ID":"b228ca84-f54d-4819-8ab9-bf4b8983ac69","Type":"ContainerStarted","Data":"58dc6bef384b6291d7b25fe7389ea28b6fd3dd097f97aa07db43584386e17358"} Jun 09 07:58:59.311044 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.311012 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="eae34d6004e890ade36978c4bac258dafa5aeeff87b7f5b15a3842f26ffbd467" exitCode=0 Jun 09 07:58:59.311193 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:58:59.311059 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"eae34d6004e890ade36978c4bac258dafa5aeeff87b7f5b15a3842f26ffbd467"} Jun 09 07:59:00.124379 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:00.124327 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:59:00.124591 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:00.124508 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:59:00.124591 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:00.124532 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:59:00.124591 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:00.124593 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:02.12457576 +0000 UTC m=+39.565039716 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:59:00.225118 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:00.225009 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:59:00.225263 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:00.225154 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:59:00.225263 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:00.225241 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:59:02.225218452 +0000 UTC m=+39.665682427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:59:00.317214 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:00.317177 2573 generic.go:356] "Generic (PLEG): container finished" podID="8dc53d82-c5ab-4356-ad54-b17575987bc3" containerID="888a194baace121765a0b90da82a00cc3cc82ab20f56786fabcab8b83a51828c" exitCode=0 Jun 09 07:59:00.317607 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:00.317238 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerDied","Data":"888a194baace121765a0b90da82a00cc3cc82ab20f56786fabcab8b83a51828c"} Jun 09 07:59:01.319550 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.319357 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:01.321682 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.321651 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jh5j9" event={"ID":"b228ca84-f54d-4819-8ab9-bf4b8983ac69","Type":"ContainerStarted","Data":"d711437030b070a5d91f795cd06e75610a51cbf6b0b0738773176a73e97f3f85"} Jun 09 07:59:01.325008 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.324975 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" event={"ID":"8dc53d82-c5ab-4356-ad54-b17575987bc3","Type":"ContainerStarted","Data":"d89673d4cb479a71068abc5cbfbe6b66d78fb3b51aa17c776a017260507c1875"} Jun 09 07:59:01.340930 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.340799 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jh5j9" podStartSLOduration=1.698742637 podStartE2EDuration="3.340785327s" totalImagesPullingTime="1.64204269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:01.340746549 +0000 UTC m=+38.781210527" watchObservedRunningTime="2026-06-09 07:59:01.340785327 +0000 UTC m=+38.781249345" Jun 09 07:59:01.368159 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.368109 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wkqzz" podStartSLOduration=8.568534791 podStartE2EDuration="38.368091244s" totalImagesPullingTime="29.799556453s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:01.36697622 +0000 UTC m=+38.807440199" watchObservedRunningTime="2026-06-09 07:59:01.368091244 +0000 UTC m=+38.808555222" Jun 09 07:59:01.421183 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.421147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:01.523714 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.523672 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:01.621501 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:01.621414 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:02.142213 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:02.142169 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:59:02.142432 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:02.142355 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:59:02.142432 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:02.142378 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:59:02.142543 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:02.142453 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:06.142430565 +0000 UTC m=+43.582894551 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:59:02.242880 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:02.242837 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:59:02.243044 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:02.242988 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:59:02.243088 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:02.243057 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:59:06.243038083 +0000 UTC m=+43.683502038 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:59:04.332018 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.331985 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4"] Jun 09 07:59:04.366696 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.366652 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4"] Jun 09 07:59:04.366855 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.366768 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.369218 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.369192 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 09 07:59:04.369338 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.369247 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 09 07:59:04.369338 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.369294 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 09 07:59:04.369643 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.369628 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 09 07:59:04.419176 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.419144 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h"] Jun 09 07:59:04.444504 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.444470 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h"] Jun 09 07:59:04.444682 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.444606 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.446698 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.446670 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 09 07:59:04.446827 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.446728 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 09 07:59:04.446895 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.446834 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 09 07:59:04.446989 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.446974 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 09 07:59:04.459113 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.459078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4021dc64-c4e2-484c-9726-c3796a71060d-tmp\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.459306 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.459163 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4021dc64-c4e2-484c-9726-c3796a71060d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.459306 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.459264 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9thl6\" (UniqueName: \"kubernetes.io/projected/4021dc64-c4e2-484c-9726-c3796a71060d-kube-api-access-9thl6\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.559638 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559598 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4021dc64-c4e2-484c-9726-c3796a71060d-tmp\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.559850 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559659 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.559850 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4021dc64-c4e2-484c-9726-c3796a71060d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.559850 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559698 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5nh7t\" (UniqueName: \"kubernetes.io/projected/e139e09d-6f7b-48e6-a9c6-a7e795af6382-kube-api-access-5nh7t\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.559850 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559718 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.559850 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559766 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.560147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559853 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9thl6\" (UniqueName: \"kubernetes.io/projected/4021dc64-c4e2-484c-9726-c3796a71060d-kube-api-access-9thl6\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.560147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559918 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.560147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.559947 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e139e09d-6f7b-48e6-a9c6-a7e795af6382-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.560147 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.560052 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4021dc64-c4e2-484c-9726-c3796a71060d-tmp\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.563746 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.563729 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/4021dc64-c4e2-484c-9726-c3796a71060d-klusterlet-config\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.568630 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.568601 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9thl6\" (UniqueName: \"kubernetes.io/projected/4021dc64-c4e2-484c-9726-c3796a71060d-kube-api-access-9thl6\") pod \"klusterlet-addon-workmgr-76b645bc-9sdx4\" (UID: \"4021dc64-c4e2-484c-9726-c3796a71060d\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.660618 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660516 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.660618 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.660618 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e139e09d-6f7b-48e6-a9c6-a7e795af6382-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.660941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660734 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.660941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660781 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5nh7t\" (UniqueName: \"kubernetes.io/projected/e139e09d-6f7b-48e6-a9c6-a7e795af6382-kube-api-access-5nh7t\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.660941 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.660815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.661367 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.661340 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/e139e09d-6f7b-48e6-a9c6-a7e795af6382-additional-service-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.663240 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.663212 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-ca\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.663379 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.663363 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.663507 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.663486 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.664009 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.663989 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/e139e09d-6f7b-48e6-a9c6-a7e795af6382-hub\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.669340 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.669313 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5nh7t\" (UniqueName: \"kubernetes.io/projected/e139e09d-6f7b-48e6-a9c6-a7e795af6382-kube-api-access-5nh7t\") pod \"cluster-proxy-proxy-agent-56774fb967-xqr8h\" (UID: \"e139e09d-6f7b-48e6-a9c6-a7e795af6382\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.675980 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.675958 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:04.765417 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.765096 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 07:59:04.845931 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.845898 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4"] Jun 09 07:59:04.851096 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:59:04.851012 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4021dc64_c4e2_484c_9726_c3796a71060d.slice/crio-d8d194d66328c9a784341a8c31ec149c3f6a15f17200dcefd8c912b16bf95a35 WatchSource:0}: Error finding container d8d194d66328c9a784341a8c31ec149c3f6a15f17200dcefd8c912b16bf95a35: Status 404 returned error can't find the container with id d8d194d66328c9a784341a8c31ec149c3f6a15f17200dcefd8c912b16bf95a35 Jun 09 07:59:04.932318 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:04.932247 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h"] Jun 09 07:59:04.935796 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:59:04.935768 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode139e09d_6f7b_48e6_a9c6_a7e795af6382.slice/crio-b3ca8bdaaf149934c93f35e22d117d553234aed302f973b56d9ed1c51cc280fb WatchSource:0}: Error finding container b3ca8bdaaf149934c93f35e22d117d553234aed302f973b56d9ed1c51cc280fb: Status 404 returned error can't find the container with id b3ca8bdaaf149934c93f35e22d117d553234aed302f973b56d9ed1c51cc280fb Jun 09 07:59:05.335709 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:05.335609 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerStarted","Data":"b3ca8bdaaf149934c93f35e22d117d553234aed302f973b56d9ed1c51cc280fb"} Jun 09 07:59:05.336657 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:05.336635 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" event={"ID":"4021dc64-c4e2-484c-9726-c3796a71060d","Type":"ContainerStarted","Data":"d8d194d66328c9a784341a8c31ec149c3f6a15f17200dcefd8c912b16bf95a35"} Jun 09 07:59:06.176024 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:06.175984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:59:06.176209 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:06.176175 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:59:06.176209 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:06.176194 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:59:06.176279 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:06.176254 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:14.17623359 +0000 UTC m=+51.616697546 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:59:06.276947 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:06.276819 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:59:06.277122 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:06.277004 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:59:06.277122 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:06.277112 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:59:14.277058619 +0000 UTC m=+51.717522581 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:59:07.161548 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:07.161510 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:07.261883 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:07.261827 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:11.356761 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.356670 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerStarted","Data":"18cc2a8b000dd24e47eb2a173d4b7118fcf6123963383d7a42c0f16bf3a31bf4"} Jun 09 07:59:11.356761 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.356722 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerStarted","Data":"994574a756e488bc69e966ac17bd9916c3badc15de3c73bf42a9ecc5d965e0bc"} Jun 09 07:59:11.356761 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.356739 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerStarted","Data":"dc4cf3d004918642460c6cf240561d79c703c89cda539c92833da169cb459fd3"} Jun 09 07:59:11.358201 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.358179 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" event={"ID":"4021dc64-c4e2-484c-9726-c3796a71060d","Type":"ContainerStarted","Data":"1247807fa02ab402878cb823128116f15079eded533a98e412bd19fd31368dd7"} Jun 09 07:59:11.358412 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.358395 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:11.360226 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.360204 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 07:59:11.391883 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.391807 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" podStartSLOduration=1.604312626 podStartE2EDuration="7.391789427s" totalImagesPullingTime="5.787476801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:11.376200111 +0000 UTC m=+48.816664089" watchObservedRunningTime="2026-06-09 07:59:11.391789427 +0000 UTC m=+48.832253426" Jun 09 07:59:11.392271 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:11.392244 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" podStartSLOduration=1.5062477090000002 podStartE2EDuration="7.392235587s" totalImagesPullingTime="5.885987878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:59:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:11.392139512 +0000 UTC m=+48.832603491" watchObservedRunningTime="2026-06-09 07:59:11.392235587 +0000 UTC m=+48.832699564" Jun 09 07:59:14.246991 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:14.246947 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:59:14.247555 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:14.247106 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:59:14.247555 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:14.247130 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:59:14.247555 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:14.247208 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 07:59:30.24718618 +0000 UTC m=+67.687650140 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:59:14.302482 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:14.302450 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:14.348089 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:14.348047 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:59:14.348278 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:14.348205 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:59:14.348278 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:14.348283 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 07:59:30.348261122 +0000 UTC m=+67.788725092 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:59:14.403916 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:14.403857 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:23.878365 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:23.878334 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ddz24" Jun 09 07:59:24.034591 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:24.034553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:24.097724 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:24.097684 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:27.744045 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.744003 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 07:59:27.746361 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.746344 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 09 07:59:27.755145 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:27.755124 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 07:59:27.755214 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:27.755182 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 08:00:31.755166941 +0000 UTC m=+129.195630897 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : secret "metrics-daemon-secret" not found Jun 09 07:59:27.844771 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.844725 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:59:27.846725 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.846712 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 09 07:59:27.857327 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.857294 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4-original-pull-secret\") pod \"global-pull-secret-syncer-glwhj\" (UID: \"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4\") " pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:59:27.946139 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.946095 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:59:27.948329 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.948312 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 09 07:59:27.958326 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.958305 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 09 07:59:27.969428 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:27.969402 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58zb8\" (UniqueName: \"kubernetes.io/projected/384ac8e6-a50f-400f-ae74-71bcd97bfdfd-kube-api-access-58zb8\") pod \"network-check-target-pbs9g\" (UID: \"384ac8e6-a50f-400f-ae74-71bcd97bfdfd\") " pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:59:28.116703 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.116613 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-glwhj" Jun 09 07:59:28.124756 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.124735 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-dscxw\"" Jun 09 07:59:28.132729 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.132707 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:59:28.244732 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.244703 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-glwhj"] Jun 09 07:59:28.248144 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:59:28.248115 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d7869c1_1935_42f4_8ff3_3b4ecdb49ff4.slice/crio-de715bc75a4bb2269df1e9a28b25b97dbf43991232ac8ad8afcfa00d538736e5 WatchSource:0}: Error finding container de715bc75a4bb2269df1e9a28b25b97dbf43991232ac8ad8afcfa00d538736e5: Status 404 returned error can't find the container with id de715bc75a4bb2269df1e9a28b25b97dbf43991232ac8ad8afcfa00d538736e5 Jun 09 07:59:28.263783 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.263756 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pbs9g"] Jun 09 07:59:28.266590 ip-10-0-141-98 kubenswrapper[2573]: W0609 07:59:28.266561 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384ac8e6_a50f_400f_ae74_71bcd97bfdfd.slice/crio-d77cbaa7baf0d25199109892d93bcaca0e01f2b01eb73891bfba398881458c6d WatchSource:0}: Error finding container d77cbaa7baf0d25199109892d93bcaca0e01f2b01eb73891bfba398881458c6d: Status 404 returned error can't find the container with id d77cbaa7baf0d25199109892d93bcaca0e01f2b01eb73891bfba398881458c6d Jun 09 07:59:28.401007 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.400906 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pbs9g" event={"ID":"384ac8e6-a50f-400f-ae74-71bcd97bfdfd","Type":"ContainerStarted","Data":"d77cbaa7baf0d25199109892d93bcaca0e01f2b01eb73891bfba398881458c6d"} Jun 09 07:59:28.401986 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:28.401956 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-glwhj" event={"ID":"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4","Type":"ContainerStarted","Data":"de715bc75a4bb2269df1e9a28b25b97dbf43991232ac8ad8afcfa00d538736e5"} Jun 09 07:59:30.264874 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:30.264818 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 07:59:30.265340 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:30.265006 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 07:59:30.265340 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:30.265030 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 07:59:30.265340 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:30.265104 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 08:00:02.265081827 +0000 UTC m=+99.705545804 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 07:59:30.365308 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:30.365263 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 07:59:30.365504 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:30.365449 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 07:59:30.365568 ip-10-0-141-98 kubenswrapper[2573]: E0609 07:59:30.365527 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 08:00:02.365505365 +0000 UTC m=+99.805969326 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 07:59:32.299936 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:32.299890 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:32.401361 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:32.401325 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 07:59:33.419292 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:33.419251 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pbs9g" event={"ID":"384ac8e6-a50f-400f-ae74-71bcd97bfdfd","Type":"ContainerStarted","Data":"31c60481db739617d38d57ae70e43d939ba158118513c860e01fefe4dd691c65"} Jun 09 07:59:33.419739 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:33.419340 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 07:59:33.420776 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:33.420741 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-glwhj" event={"ID":"9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4","Type":"ContainerStarted","Data":"9e75b96f20a92dfacf9bfa533e8945aa7503f3da23c8103fc89ed6162644ab08"} Jun 09 07:59:33.437017 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:33.436968 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pbs9g" podStartSLOduration=66.162289007 podStartE2EDuration="1m10.436953357s" totalImagesPullingTime="4.27466435s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:33.436288529 +0000 UTC m=+70.876752502" watchObservedRunningTime="2026-06-09 07:59:33.436953357 +0000 UTC m=+70.877417336" Jun 09 07:59:33.451084 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:33.451035 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-glwhj" podStartSLOduration=66.15535396 podStartE2EDuration="1m10.451019096s" totalImagesPullingTime="4.295665136s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 07:59:33.45100917 +0000 UTC m=+70.891473148" watchObservedRunningTime="2026-06-09 07:59:33.451019096 +0000 UTC m=+70.891483071" Jun 09 07:59:52.178593 ip-10-0-141-98 kubenswrapper[2573]: I0609 07:59:52.178542 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:00:02.299530 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:00:02.299495 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:00:02.299968 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:02.299641 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 08:00:02.299968 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:02.299658 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 08:00:02.299968 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:02.299722 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 08:01:06.299706147 +0000 UTC m=+163.740170124 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 08:00:02.400424 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:00:02.400386 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 08:00:02.400594 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:02.400547 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 08:00:02.400635 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:02.400615 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 08:01:06.400596753 +0000 UTC m=+163.841060730 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 08:00:04.427189 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:00:04.427158 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pbs9g" Jun 09 08:00:31.814080 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:00:31.814038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 08:00:31.814566 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:31.814193 2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 09 08:00:31.814566 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:00:31.814265 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs podName:e1e16526-f4bc-4e20-abb0-999881c8dcc4 nodeName:}" failed. No retries permitted until 2026-06-09 08:02:33.814247601 +0000 UTC m=+251.254711557 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs") pod "network-metrics-daemon-h55kx" (UID: "e1e16526-f4bc-4e20-abb0-999881c8dcc4") : secret "metrics-daemon-secret" not found Jun 09 08:01:01.444629 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:01.444584 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" podUID="cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" Jun 09 08:01:01.464729 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:01.464696 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5v8n5" podUID="cd8b2f40-3541-4e14-906b-440cb76c416d" Jun 09 08:01:01.676158 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:01.676126 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:01:01.676321 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:01.676168 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5v8n5" Jun 09 08:01:02.027117 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:02.027055 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-h55kx" podUID="e1e16526-f4bc-4e20-abb0-999881c8dcc4" Jun 09 08:01:06.019684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:06.019648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:06.361533 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:06.361447 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") pod \"image-registry-7f55f497b5-4bpqs\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:01:06.361751 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:06.361604 2573 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 09 08:01:06.361751 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:06.361624 2573 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f55f497b5-4bpqs: secret "image-registry-tls" not found Jun 09 08:01:06.361751 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:06.361711 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls podName:cc886b0f-2b59-4da5-9d7e-6ae241ed0de2 nodeName:}" failed. No retries permitted until 2026-06-09 08:03:08.36169479 +0000 UTC m=+285.802158749 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls") pod "image-registry-7f55f497b5-4bpqs" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2") : secret "image-registry-tls" not found Jun 09 08:01:06.462403 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:06.462363 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 08:01:06.462554 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:06.462533 2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 09 08:01:06.462625 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:01:06.462614 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls podName:cd8b2f40-3541-4e14-906b-440cb76c416d nodeName:}" failed. No retries permitted until 2026-06-09 08:03:08.462599059 +0000 UTC m=+285.903063031 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls") pod "dns-default-5v8n5" (UID: "cd8b2f40-3541-4e14-906b-440cb76c416d") : secret "dns-default-metrics-tls" not found Jun 09 08:01:11.359494 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:11.359441 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" podUID="4021dc64-c4e2-484c-9726-c3796a71060d" containerName="acm-agent" probeResult="failure" output="Get \"http://10.133.0.9:8000/readyz\": dial tcp 10.133.0.9:8000: connect: connection refused" Jun 09 08:01:11.707560 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:11.707527 2573 generic.go:356] "Generic (PLEG): container finished" podID="4021dc64-c4e2-484c-9726-c3796a71060d" containerID="1247807fa02ab402878cb823128116f15079eded533a98e412bd19fd31368dd7" exitCode=1 Jun 09 08:01:11.707816 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:11.707604 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" event={"ID":"4021dc64-c4e2-484c-9726-c3796a71060d","Type":"ContainerDied","Data":"1247807fa02ab402878cb823128116f15079eded533a98e412bd19fd31368dd7"} Jun 09 08:01:11.707958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:11.707944 2573 scope.go:117] "RemoveContainer" containerID="1247807fa02ab402878cb823128116f15079eded533a98e412bd19fd31368dd7" Jun 09 08:01:12.712476 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:12.712437 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" event={"ID":"4021dc64-c4e2-484c-9726-c3796a71060d","Type":"ContainerStarted","Data":"96bfafca30a9eaebe948c35b9d8b2f56f36e6167cab3f074043b88c822a253fa"} Jun 09 08:01:12.712932 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:12.712721 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 08:01:12.713328 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:12.713311 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-76b645bc-9sdx4" Jun 09 08:01:14.005972 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:14.005940 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 08:01:54.456107 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:54.456070 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:54.557366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:54.557334 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:56.669841 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:56.669810 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:56.770606 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:56.770575 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.128554 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.128517 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.240196 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.240162 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.335016 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.334984 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.441911 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.441847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.550677 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.550634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.606369 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.606340 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.828690 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.828596 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:01:59.928945 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:01:59.928908 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:01.469566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:01.469529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:01.519520 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:01.519487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:01.692439 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:01.692405 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:01.749153 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:01.749070 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:01.971441 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:01.971417 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jc2tq_ed44772a-57dc-425a-8834-266066f4a961/dns-node-resolver/0.log" Jun 09 08:02:02.398092 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:02.398057 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:02.797189 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:02.797165 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.074155 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.074051 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.186064 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.186014 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.197903 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.197883 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p89d8_2c562ebb-1fe5-491b-b3de-e5926b9d4cee/node-ca/0.log" Jun 09 08:02:03.274331 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.274278 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.375086 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.374993 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.706315 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.706282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.905285 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.905245 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:03.971149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:03.971075 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jh5j9_b228ca84-f54d-4819-8ab9-bf4b8983ac69/serve-healthcheck-canary/0.log" Jun 09 08:02:04.911234 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:04.911200 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:05.302719 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:05.302685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:05.497233 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:05.497201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:05.706127 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:05.706092 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:05.917049 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:05.917013 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:06.324594 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:06.324563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:06.704212 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:06.704178 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:06.839495 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:06.839459 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:07.102731 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:07.102648 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:07.331229 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:07.331194 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:07.501879 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:07.501845 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:07.701453 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:07.701426 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:07.904646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:07.904561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:08.918270 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:08.918228 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:09.310618 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:09.310584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:09.398713 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:09.398683 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:09.515075 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:09.515046 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:09.703704 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:09.703669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:09.913659 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:09.913631 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:10.899913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:10.899852 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:11.298163 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:11.298125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:11.502480 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:11.502449 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:11.694545 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:11.694515 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:11.904257 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:11.904221 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:12.404384 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:12.404355 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:12.697319 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:12.697287 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:12.904497 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:12.904464 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:13.105091 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:13.105000 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:13.332807 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:13.332773 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:13.400087 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:13.400017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:13.499963 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:13.499931 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:13.656444 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:13.656367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:25.653564 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.653530 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvb4"] Jun 09 08:02:25.656479 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.656458 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.658548 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.658523 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 09 08:02:25.658663 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.658523 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 09 08:02:25.659146 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.659126 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 09 08:02:25.663306 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.659784 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8gl6z\"" Jun 09 08:02:25.663306 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.659808 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 09 08:02:25.668787 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.668761 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvb4"] Jun 09 08:02:25.726140 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.726102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-data-volume\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.726140 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.726140 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.726376 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.726176 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.726376 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.726262 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzm9g\" (UniqueName: \"kubernetes.io/projected/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-api-access-dzm9g\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.726376 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.726311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-crio-socket\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.741805 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.741776 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f55f497b5-4bpqs"] Jun 09 08:02:25.742017 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:02:25.741996 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" podUID="cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" Jun 09 08:02:25.827461 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827423 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-data-volume\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827667 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827471 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827667 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827515 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827667 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827563 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzm9g\" (UniqueName: \"kubernetes.io/projected/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-api-access-dzm9g\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827667 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827601 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-crio-socket\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827838 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827688 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-crio-socket\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.827838 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.827806 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-data-volume\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.828075 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.828056 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.829910 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.829891 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.837613 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.837581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:25.838693 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.838671 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzm9g\" (UniqueName: \"kubernetes.io/projected/0042e01a-f1ce-4e6e-be6e-1503350cd4e1-kube-api-access-dzm9g\") pod \"insights-runtime-extractor-rsvb4\" (UID: \"0042e01a-f1ce-4e6e-be6e-1503350cd4e1\") " pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:25.919850 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.919772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:02:25.926616 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.926587 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:02:25.941339 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.941293 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:25.967693 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:25.967655 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rsvb4" Jun 09 08:02:26.029741 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029705 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.029944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029760 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7n2f4\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.029944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029810 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.029944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029859 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.029944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029914 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.030164 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.029948 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.030164 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.030005 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration\") pod \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\" (UID: \"cc886b0f-2b59-4da5-9d7e-6ae241ed0de2\") " Jun 09 08:02:26.032128 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.031637 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:02:26.032128 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.031765 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 08:02:26.032128 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.031797 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:02:26.033377 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.033346 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:02:26.033635 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.033606 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:02:26.034029 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.034005 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4" (OuterVolumeSpecName: "kube-api-access-7n2f4") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "kube-api-access-7n2f4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:02:26.034241 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.034213 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" (UID: "cc886b0f-2b59-4da5-9d7e-6ae241ed0de2"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:02:26.094894 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.094846 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rsvb4"] Jun 09 08:02:26.098016 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:02:26.097982 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0042e01a_f1ce_4e6e_be6e_1503350cd4e1.slice/crio-dbbc9950927184456b77c74a47814e07106127e050b7ea304824418083ec51a9 WatchSource:0}: Error finding container dbbc9950927184456b77c74a47814e07106127e050b7ea304824418083ec51a9: Status 404 returned error can't find the container with id dbbc9950927184456b77c74a47814e07106127e050b7ea304824418083ec51a9 Jun 09 08:02:26.130727 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130698 2573 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-installation-pull-secrets\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130727 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130729 2573 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-trusted-ca\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130740 2573 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-certificates\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130749 2573 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-ca-trust-extracted\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130759 2573 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-image-registry-private-configuration\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130769 2573 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-bound-sa-token\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.130913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.130777 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7n2f4\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-kube-api-access-7n2f4\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:26.924323 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.924282 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvb4" event={"ID":"0042e01a-f1ce-4e6e-be6e-1503350cd4e1","Type":"ContainerStarted","Data":"f04dc8679e60fc83ace491f3b8a435af85ace17f24de2ac37480e2fb174afff9"} Jun 09 08:02:26.924718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.924353 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvb4" event={"ID":"0042e01a-f1ce-4e6e-be6e-1503350cd4e1","Type":"ContainerStarted","Data":"df0961f969fb6d11993cd3497d506c5fc9f6ecb6aa2cbf5b6c7738fcd5185af3"} Jun 09 08:02:26.924718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.924376 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvb4" event={"ID":"0042e01a-f1ce-4e6e-be6e-1503350cd4e1","Type":"ContainerStarted","Data":"dbbc9950927184456b77c74a47814e07106127e050b7ea304824418083ec51a9"} Jun 09 08:02:26.924718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.924296 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f55f497b5-4bpqs" Jun 09 08:02:26.960153 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.960117 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f55f497b5-4bpqs"] Jun 09 08:02:26.965048 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:26.965021 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f55f497b5-4bpqs"] Jun 09 08:02:27.008830 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:27.008788 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc886b0f-2b59-4da5-9d7e-6ae241ed0de2" path="/var/lib/kubelet/pods/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2/volumes" Jun 09 08:02:27.037775 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:27.037696 2573 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/cc886b0f-2b59-4da5-9d7e-6ae241ed0de2-registry-tls\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:02:28.835210 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:28.835174 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:28.947608 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:28.947561 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.016192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.016147 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.411787 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.411698 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.513488 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.513450 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.597648 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.597612 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.789082 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.789048 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:29.934825 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.934789 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rsvb4" event={"ID":"0042e01a-f1ce-4e6e-be6e-1503350cd4e1","Type":"ContainerStarted","Data":"6a38521bb7cae843f26b49e39a0afa067103b447b8f8fc3ff29c94f27f29e3a3"} Jun 09 08:02:29.953637 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:29.953583 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rsvb4" podStartSLOduration=2.162747564 podStartE2EDuration="4.953565671s" totalImagesPullingTime="2.790818107s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 08:02:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:02:29.953128376 +0000 UTC m=+247.393592354" watchObservedRunningTime="2026-06-09 08:02:29.953565671 +0000 UTC m=+247.394029648" Jun 09 08:02:33.600768 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:33.600729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:33.668774 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:33.668735 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:33.890563 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:33.890466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 08:02:33.892795 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:33.892773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e1e16526-f4bc-4e20-abb0-999881c8dcc4-metrics-certs\") pod \"network-metrics-daemon-h55kx\" (UID: \"e1e16526-f4bc-4e20-abb0-999881c8dcc4\") " pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 08:02:34.109229 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:34.109199 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n7k9q\"" Jun 09 08:02:34.117899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:34.117847 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-h55kx" Jun 09 08:02:34.246292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:34.246258 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-h55kx"] Jun 09 08:02:34.250366 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:02:34.250328 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1e16526_f4bc_4e20_abb0_999881c8dcc4.slice/crio-76d467d3cf27c253e6a5dfe3d4a18cb5b86ad959977bbbc238cde7d5c94dcd4c WatchSource:0}: Error finding container 76d467d3cf27c253e6a5dfe3d4a18cb5b86ad959977bbbc238cde7d5c94dcd4c: Status 404 returned error can't find the container with id 76d467d3cf27c253e6a5dfe3d4a18cb5b86ad959977bbbc238cde7d5c94dcd4c Jun 09 08:02:34.418662 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:34.418576 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:34.953661 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:34.953628 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55kx" event={"ID":"e1e16526-f4bc-4e20-abb0-999881c8dcc4","Type":"ContainerStarted","Data":"76d467d3cf27c253e6a5dfe3d4a18cb5b86ad959977bbbc238cde7d5c94dcd4c"} Jun 09 08:02:35.960497 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:35.960458 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55kx" event={"ID":"e1e16526-f4bc-4e20-abb0-999881c8dcc4","Type":"ContainerStarted","Data":"0332c0d4e86af9cc497875f18c0109f805b1dc15e557a3d7d50a2cd626e92a93"} Jun 09 08:02:35.960941 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:35.960516 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-h55kx" event={"ID":"e1e16526-f4bc-4e20-abb0-999881c8dcc4","Type":"ContainerStarted","Data":"63824f1d484757fca6cfda4541e4ca48d015f4ed21c39db014af22283e010a3b"} Jun 09 08:02:35.981614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:35.981563 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-h55kx" podStartSLOduration=252.037935868 podStartE2EDuration="4m12.981547429s" totalImagesPullingTime="943.611561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:02:35.98050468 +0000 UTC m=+253.420968658" watchObservedRunningTime="2026-06-09 08:02:35.981547429 +0000 UTC m=+253.422011427" Jun 09 08:02:36.087858 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:36.087826 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:36.143785 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:36.143753 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:36.197808 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:36.197777 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:37.087278 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:37.087240 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:37.910416 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:37.910377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:38.011437 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:38.011390 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.040705 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.040661 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.142975 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.142938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.219521 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.219484 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.351109 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.351017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.760523 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.760487 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.855049 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.855020 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:39.954693 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:39.954650 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.056259 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.056169 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.090031 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.089996 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-mstf4"] Jun 09 08:02:40.093011 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.092992 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.094839 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.094817 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 09 08:02:40.095137 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095121 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 09 08:02:40.095347 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095329 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 09 08:02:40.095398 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095335 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 09 08:02:40.095842 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095826 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-qzrdc\"" Jun 09 08:02:40.095950 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095934 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 09 08:02:40.096003 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.095954 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 09 08:02:40.138261 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138222 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-root\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138436 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138265 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138436 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138303 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-metrics-client-ca\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138436 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138333 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5t8pm\" (UniqueName: \"kubernetes.io/projected/b121b8f8-011d-4715-b3e2-e6e2563f28a1-kube-api-access-5t8pm\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138536 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138449 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-wtmp\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138536 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138478 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-sys\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138536 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138497 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-tls\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138536 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138515 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-textfile\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.138670 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.138614 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239015 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.238972 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-wtmp\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239015 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239016 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-sys\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-tls\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239056 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-textfile\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239114 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-sys\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239143 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-wtmp\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239204 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239242 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-root\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239265 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239289 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-metrics-client-ca\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239312 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5t8pm\" (UniqueName: \"kubernetes.io/projected/b121b8f8-011d-4715-b3e2-e6e2563f28a1-kube-api-access-5t8pm\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239345 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/b121b8f8-011d-4715-b3e2-e6e2563f28a1-root\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239390 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-textfile\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239782 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239751 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-accelerators-collector-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.239973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.239955 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b121b8f8-011d-4715-b3e2-e6e2563f28a1-metrics-client-ca\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.242117 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.242098 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-tls\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.242221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.242200 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/b121b8f8-011d-4715-b3e2-e6e2563f28a1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.255735 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.255713 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5t8pm\" (UniqueName: \"kubernetes.io/projected/b121b8f8-011d-4715-b3e2-e6e2563f28a1-kube-api-access-5t8pm\") pod \"node-exporter-mstf4\" (UID: \"b121b8f8-011d-4715-b3e2-e6e2563f28a1\") " pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.260739 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.260708 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.361444 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.361353 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.402504 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.402468 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-mstf4" Jun 09 08:02:40.412255 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:02:40.412228 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb121b8f8_011d_4715_b3e2_e6e2563f28a1.slice/crio-cae6a26da967037b1d353469d64447868be0898acf38eb096b708e543c2dd771 WatchSource:0}: Error finding container cae6a26da967037b1d353469d64447868be0898acf38eb096b708e543c2dd771: Status 404 returned error can't find the container with id cae6a26da967037b1d353469d64447868be0898acf38eb096b708e543c2dd771 Jun 09 08:02:40.462651 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.462596 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.566237 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.566201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.666069 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.665961 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.794916 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.794885 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.916891 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.916773 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:40.980442 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:40.980401 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mstf4" event={"ID":"b121b8f8-011d-4715-b3e2-e6e2563f28a1","Type":"ContainerStarted","Data":"cae6a26da967037b1d353469d64447868be0898acf38eb096b708e543c2dd771"} Jun 09 08:02:41.031421 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.031371 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:41.131180 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.131144 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:02:41.135047 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.135013 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.135711 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.135643 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:41.137011 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.136988 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jun 09 08:02:41.138087 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138016 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jun 09 08:02:41.138187 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138160 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jun 09 08:02:41.138252 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138181 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jun 09 08:02:41.138252 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138210 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jun 09 08:02:41.138403 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138384 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jun 09 08:02:41.138514 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138491 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jun 09 08:02:41.138575 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138512 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jun 09 08:02:41.138882 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138845 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jun 09 08:02:41.138983 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.138899 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jffzh\"" Jun 09 08:02:41.157741 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.156966 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:02:41.235684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.235638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:41.246760 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246721 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgvms\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.246946 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246767 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.246946 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.246946 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246875 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.246946 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246916 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247138 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.246955 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247138 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247051 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247138 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247096 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247253 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247161 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247253 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247204 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247253 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247241 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247368 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247266 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.247368 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.247297 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348522 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348492 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348531 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgvms\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348557 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348573 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348591 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348607 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348627 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348652 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348680 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348724 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348762 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:02:41.348770 2573 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348794 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.348814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.348978 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:02:41.348889 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls podName:b5f52318-eb86-4d9c-99b9-75540b99b35f nodeName:}" failed. No retries permitted until 2026-06-09 08:02:41.8488298 +0000 UTC m=+259.289293764 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f") : secret "alertmanager-main-tls" not found Jun 09 08:02:41.349428 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:02:41.348991 2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle podName:b5f52318-eb86-4d9c-99b9-75540b99b35f nodeName:}" failed. No retries permitted until 2026-06-09 08:02:41.848978055 +0000 UTC m=+259.289442012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f") : configmap references non-existent config key: ca-bundle.crt Jun 09 08:02:41.349428 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.349170 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.350079 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.350027 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.351676 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.351646 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.352636 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.352605 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:41.352895 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.352875 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.352956 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.352880 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.353110 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.353078 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.353214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.353196 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.353362 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.353329 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.353603 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.353586 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.353669 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.353593 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.368580 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.368551 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgvms\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.462294 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.462255 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:41.852296 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.852252 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.852621 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.852306 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.853292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.853262 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.854677 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.854656 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:41.985348 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.985311 2573 generic.go:356] "Generic (PLEG): container finished" podID="b121b8f8-011d-4715-b3e2-e6e2563f28a1" containerID="134cf2b582a6f1e661ba121ccf59466ed800520598b53acd6a539e4a21dc363d" exitCode=0 Jun 09 08:02:41.985518 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:41.985352 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mstf4" event={"ID":"b121b8f8-011d-4715-b3e2-e6e2563f28a1","Type":"ContainerDied","Data":"134cf2b582a6f1e661ba121ccf59466ed800520598b53acd6a539e4a21dc363d"} Jun 09 08:02:42.045940 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.045914 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:02:42.188384 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.188347 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:02:42.192520 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:02:42.192497 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5f52318_eb86_4d9c_99b9_75540b99b35f.slice/crio-2d02213eaac1ae3c8c2f2a0e249e7d4238d0823cadc196defd53e67f5d41759f WatchSource:0}: Error finding container 2d02213eaac1ae3c8c2f2a0e249e7d4238d0823cadc196defd53e67f5d41759f: Status 404 returned error can't find the container with id 2d02213eaac1ae3c8c2f2a0e249e7d4238d0823cadc196defd53e67f5d41759f Jun 09 08:02:42.968139 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.968096 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:42.991355 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.991318 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mstf4" event={"ID":"b121b8f8-011d-4715-b3e2-e6e2563f28a1","Type":"ContainerStarted","Data":"9d3e4b17effe1ed7a02cacc93e6c6d933081b9012988401a6623f32c0bb9b849"} Jun 09 08:02:42.991547 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.991382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-mstf4" event={"ID":"b121b8f8-011d-4715-b3e2-e6e2563f28a1","Type":"ContainerStarted","Data":"7ac8d3733780227e5819eb00d1bec5a4db411cbb06dbd048eb2cfb902a6c7372"} Jun 09 08:02:42.992609 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:42.992582 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"2d02213eaac1ae3c8c2f2a0e249e7d4238d0823cadc196defd53e67f5d41759f"} Jun 09 08:02:43.010804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.010749 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-mstf4" podStartSLOduration=2.052272392 podStartE2EDuration="3.010729155s" totalImagesPullingTime="893.060437ms" totalInitContainerRuntime="65.396326ms" isStatefulPod=true podCreationTimestamp="2026-06-09 08:02:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:02:43.009406678 +0000 UTC m=+260.449870657" watchObservedRunningTime="2026-06-09 08:02:43.010729155 +0000 UTC m=+260.451193133" Jun 09 08:02:43.066422 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.066389 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:43.162294 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.162260 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:43.231164 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.231127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:43.331397 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.331367 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:43.997772 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.997733 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2" exitCode=0 Jun 09 08:02:43.997961 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:43.997820 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2"} Jun 09 08:02:44.789836 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:44.789798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:44.882330 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:44.882295 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:45.115919 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:45.115804 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:46.009048 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.009022 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3"} Jun 09 08:02:46.009388 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.009093 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250"} Jun 09 08:02:46.009388 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.009121 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6"} Jun 09 08:02:46.009388 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.009154 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0"} Jun 09 08:02:46.287622 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.287529 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:46.289791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.289765 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:02:46.293235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.293217 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.295564 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.295536 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 08:02:46.295765 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.295740 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cg4b041n15gig\"" Jun 09 08:02:46.296325 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296301 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 08:02:46.296458 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296333 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 08:02:46.296458 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296366 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 08:02:46.296458 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296337 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 08:02:46.296458 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296405 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 08:02:46.296706 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296614 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 08:02:46.296828 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296813 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qkrcw\"" Jun 09 08:02:46.296945 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296888 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 08:02:46.296945 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296926 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 08:02:46.297071 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296929 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 08:02:46.297071 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.296983 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 08:02:46.297071 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.297010 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 08:02:46.297273 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.297255 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 08:02:46.300006 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.299985 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 08:02:46.306449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.306425 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 08:02:46.311756 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.311729 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:02:46.390023 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.389976 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390023 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390022 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390049 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390102 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390131 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390162 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390183 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390199 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390214 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390234 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390307 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390338 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390356 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390390 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390427 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390470 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390487 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntfqh\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390540 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.390614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.390565 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491635 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491692 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491736 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491765 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491792 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491819 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.491898 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491850 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491920 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491950 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.491975 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492011 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492038 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492075 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492103 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntfqh\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492135 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492168 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492207 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492755 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492248 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492755 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492283 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492755 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492310 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492755 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492587 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.492755 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.492632 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.493042 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.493013 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.493245 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.493221 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.493994 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.493971 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.494480 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.494106 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.494972 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.494947 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.495210 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.495183 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.495469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.495441 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.495547 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.495443 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.496079 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.496054 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.496660 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.496635 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.496933 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.496908 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.497412 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.497383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.497551 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.497530 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.497793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.497773 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.498372 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.498206 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.498491 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.498469 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.498883 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.498848 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.502292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.502269 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntfqh\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh\") pod \"prometheus-k8s-0\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.606989 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.606896 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:46.764054 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:46.764017 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:02:46.784097 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:02:46.784067 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod374b54b8_ce26_4537_bb88_73b9008f84f8.slice/crio-e670cfec41620f2ba44482adf87c46f96c041fe8eb946c9f993201d62bde110c WatchSource:0}: Error finding container e670cfec41620f2ba44482adf87c46f96c041fe8eb946c9f993201d62bde110c: Status 404 returned error can't find the container with id e670cfec41620f2ba44482adf87c46f96c041fe8eb946c9f993201d62bde110c Jun 09 08:02:47.013726 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.013689 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" exitCode=0 Jun 09 08:02:47.014140 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.013769 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} Jun 09 08:02:47.014140 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.013835 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"e670cfec41620f2ba44482adf87c46f96c041fe8eb946c9f993201d62bde110c"} Jun 09 08:02:47.017208 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.017103 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0"} Jun 09 08:02:47.017208 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.017157 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerStarted","Data":"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939"} Jun 09 08:02:47.084061 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:47.083921 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.535216855 podStartE2EDuration="6.08389093s" totalImagesPullingTime="3.531158183s" totalInitContainerRuntime="17.515892ms" isStatefulPod=false podCreationTimestamp="2026-06-09 08:02:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:02:47.082432277 +0000 UTC m=+264.522896251" watchObservedRunningTime="2026-06-09 08:02:47.08389093 +0000 UTC m=+264.524354900" Jun 09 08:02:51.038087 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:51.038039 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} Jun 09 08:02:51.038508 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:51.038127 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} Jun 09 08:02:53.049317 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:53.049281 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} Jun 09 08:02:53.049803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:53.049360 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} Jun 09 08:02:53.049803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:53.049405 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} Jun 09 08:02:53.049803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:53.049440 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerStarted","Data":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} Jun 09 08:02:53.082904 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:53.082565 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.922159582 podStartE2EDuration="7.082542247s" totalImagesPullingTime="5.139735519s" totalInitContainerRuntime="20.647146ms" isStatefulPod=false podCreationTimestamp="2026-06-09 08:02:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:02:53.080729223 +0000 UTC m=+270.521193212" watchObservedRunningTime="2026-06-09 08:02:53.082542247 +0000 UTC m=+270.523006227" Jun 09 08:02:55.501061 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:55.501022 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:56.577766 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:56.577722 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:56.607697 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:56.607657 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:02:57.862690 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:57.862653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:57.930264 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:57.930223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:58.059175 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:58.059144 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:02:58.122291 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:02:58.122192 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:00.352613 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:00.352553 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:00.450593 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:00.450565 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:00.944033 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:00.943992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:01.045704 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:01.045664 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:04.640080 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:04.640040 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:04.677349 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:03:04.677295 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5v8n5" podUID="cd8b2f40-3541-4e14-906b-440cb76c416d" Jun 09 08:03:04.746498 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:04.746444 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:05.088628 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:05.088595 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:05.959544 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:05.959510 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:06.029077 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:06.029039 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:07.372605 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:07.372563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:07.471079 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:07.471043 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:08.020335 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.020299 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:08.488857 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.488821 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:08.491266 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.491241 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cd8b2f40-3541-4e14-906b-440cb76c416d-metrics-tls\") pod \"dns-default-5v8n5\" (UID: \"cd8b2f40-3541-4e14-906b-440cb76c416d\") " pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:08.691961 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.691920 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-9bx48\"" Jun 09 08:03:08.700820 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.700777 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:08.737711 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.737666 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:08.800442 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.800410 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:08.827680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:08.827651 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5v8n5"] Jun 09 08:03:08.830615 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:03:08.830576 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd8b2f40_3541_4e14_906b_440cb76c416d.slice/crio-d58bf10fce734cc7993fedb46a36d8e33d0280f608f969094a37eb6c7de358b9 WatchSource:0}: Error finding container d58bf10fce734cc7993fedb46a36d8e33d0280f608f969094a37eb6c7de358b9: Status 404 returned error can't find the container with id d58bf10fce734cc7993fedb46a36d8e33d0280f608f969094a37eb6c7de358b9 Jun 09 08:03:09.103147 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:09.103051 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5v8n5" event={"ID":"cd8b2f40-3541-4e14-906b-440cb76c416d","Type":"ContainerStarted","Data":"d58bf10fce734cc7993fedb46a36d8e33d0280f608f969094a37eb6c7de358b9"} Jun 09 08:03:10.137258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:10.137215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:10.188853 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:10.188813 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:11.112022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.111986 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5v8n5" event={"ID":"cd8b2f40-3541-4e14-906b-440cb76c416d","Type":"ContainerStarted","Data":"78939219973d9b70451339e18978769216c9aaf0b8f3bec58d905be6ffe17760"} Jun 09 08:03:11.112176 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.112052 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5v8n5" event={"ID":"cd8b2f40-3541-4e14-906b-440cb76c416d","Type":"ContainerStarted","Data":"706a968d063f969ed32cddabd3e99535baeda2e90b956bfa0b7fb420f360a3dc"} Jun 09 08:03:11.112176 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.112100 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:11.133547 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.133494 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5v8n5" podStartSLOduration=251.392034013 podStartE2EDuration="4m13.133477385s" totalImagesPullingTime="1.741443372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 07:58:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:03:11.132610664 +0000 UTC m=+288.573074642" watchObservedRunningTime="2026-06-09 08:03:11.133477385 +0000 UTC m=+288.573941362" Jun 09 08:03:11.148528 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.148500 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:11.200686 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:11.200659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:13.539588 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:13.539549 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:13.641912 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:13.641881 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:14.767337 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:14.767297 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" podUID="e139e09d-6f7b-48e6-a9c6-a7e795af6382" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 08:03:20.029455 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:20.029420 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:20.130528 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:20.130490 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:20.185985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:20.185950 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:21.117369 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:21.117339 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5v8n5" Jun 09 08:03:22.953820 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:22.953788 2573 kubelet.go:1648] "Image garbage collection succeeded" Jun 09 08:03:24.767064 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:24.767024 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" podUID="e139e09d-6f7b-48e6-a9c6-a7e795af6382" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 08:03:25.736130 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:25.736100 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5v8n5_cd8b2f40-3541-4e14-906b-440cb76c416d/dns/0.log" Jun 09 08:03:25.740770 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:25.740742 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5v8n5_cd8b2f40-3541-4e14-906b-440cb76c416d/kube-rbac-proxy/0.log" Jun 09 08:03:25.902108 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:25.902081 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jc2tq_ed44772a-57dc-425a-8834-266066f4a961/dns-node-resolver/0.log" Jun 09 08:03:28.362847 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:28.362796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:28.415601 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:28.415563 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:30.301571 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:30.301536 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:30.403021 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:30.402987 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:34.767360 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:34.767318 2573 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" podUID="e139e09d-6f7b-48e6-a9c6-a7e795af6382" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 09 08:03:34.767720 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:34.767392 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" Jun 09 08:03:34.768001 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:34.767958 2573 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"18cc2a8b000dd24e47eb2a173d4b7118fcf6123963383d7a42c0f16bf3a31bf4"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 09 08:03:34.768048 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:34.768024 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" podUID="e139e09d-6f7b-48e6-a9c6-a7e795af6382" containerName="service-proxy" containerID="cri-o://18cc2a8b000dd24e47eb2a173d4b7118fcf6123963383d7a42c0f16bf3a31bf4" gracePeriod=30 Jun 09 08:03:35.207312 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:35.207282 2573 generic.go:356] "Generic (PLEG): container finished" podID="e139e09d-6f7b-48e6-a9c6-a7e795af6382" containerID="18cc2a8b000dd24e47eb2a173d4b7118fcf6123963383d7a42c0f16bf3a31bf4" exitCode=2 Jun 09 08:03:35.207478 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:35.207347 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerDied","Data":"18cc2a8b000dd24e47eb2a173d4b7118fcf6123963383d7a42c0f16bf3a31bf4"} Jun 09 08:03:35.207478 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:35.207383 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-56774fb967-xqr8h" event={"ID":"e139e09d-6f7b-48e6-a9c6-a7e795af6382","Type":"ContainerStarted","Data":"beeda69325b4ad13e8e15d33bc563c8e69ca802391e0a86e63827fb9c0ca6859"} Jun 09 08:03:39.260608 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:39.260573 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:39.316753 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:39.316719 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:46.607112 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:46.607067 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:03:46.629141 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:46.629116 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:03:47.267842 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:47.267814 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:03:59.253854 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:59.253812 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:03:59.681911 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:03:59.681879 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:00.317353 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317315 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:04:00.317718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317608 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="alertmanager" containerID="cri-o://434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" gracePeriod=120 Jun 09 08:04:00.317718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317671 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-metric" containerID="cri-o://42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" gracePeriod=120 Jun 09 08:04:00.317718 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317687 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-web" containerID="cri-o://5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" gracePeriod=120 Jun 09 08:04:00.317899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317717 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="config-reloader" containerID="cri-o://c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" gracePeriod=120 Jun 09 08:04:00.317899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317775 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="prom-label-proxy" containerID="cri-o://300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" gracePeriod=120 Jun 09 08:04:00.317899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:00.317702 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy" containerID="cri-o://a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" gracePeriod=120 Jun 09 08:04:01.303947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.303914 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" exitCode=0 Jun 09 08:04:01.303947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.303940 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" exitCode=0 Jun 09 08:04:01.303947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.303946 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" exitCode=0 Jun 09 08:04:01.303947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.303952 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" exitCode=0 Jun 09 08:04:01.304304 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.303995 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939"} Jun 09 08:04:01.304304 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.304078 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3"} Jun 09 08:04:01.304304 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.304158 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6"} Jun 09 08:04:01.304304 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.304206 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0"} Jun 09 08:04:01.556630 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.556576 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:04:01.714265 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714224 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714265 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714268 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714496 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714315 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgvms\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714496 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714340 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714496 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714376 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714656 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714515 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714656 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714585 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714656 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714634 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.714815 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.714660 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.715178 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715150 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.715288 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715215 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.715288 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715248 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.715288 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715284 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") pod \"b5f52318-eb86-4d9c-99b9-75540b99b35f\" (UID: \"b5f52318-eb86-4d9c-99b9-75540b99b35f\") " Jun 09 08:04:01.715443 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715325 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:01.715540 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715521 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.716129 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.715728 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:01.716129 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.716118 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 08:04:01.718240 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718213 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms" (OuterVolumeSpecName: "kube-api-access-kgvms") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "kube-api-access-kgvms". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:04:01.718356 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718306 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.718529 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718496 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.718671 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718640 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.718785 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718761 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out" (OuterVolumeSpecName: "config-out") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 08:04:01.718841 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718825 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:04:01.718947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718928 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume" (OuterVolumeSpecName: "config-volume") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.719008 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.718953 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.721692 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.721594 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.727202 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.727180 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config" (OuterVolumeSpecName: "web-config") pod "b5f52318-eb86-4d9c-99b9-75540b99b35f" (UID: "b5f52318-eb86-4d9c-99b9-75540b99b35f"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816737 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816782 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816795 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816805 2573 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-volume\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816814 2573 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b5f52318-eb86-4d9c-99b9-75540b99b35f-metrics-client-ca\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816822 2573 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-alertmanager-main-db\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816831 2573 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-secret-alertmanager-main-tls\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816839 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-web-config\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.816833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816848 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-tls-assets\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.817238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816856 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgvms\" (UniqueName: \"kubernetes.io/projected/b5f52318-eb86-4d9c-99b9-75540b99b35f-kube-api-access-kgvms\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.817238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816888 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b5f52318-eb86-4d9c-99b9-75540b99b35f-config-out\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:01.817238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:01.816896 2573 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/b5f52318-eb86-4d9c-99b9-75540b99b35f-cluster-tls-config\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:02.314364 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314327 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" exitCode=0 Jun 09 08:04:02.314364 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314361 2573 generic.go:356] "Generic (PLEG): container finished" podID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerID="5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" exitCode=0 Jun 09 08:04:02.314550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314411 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0"} Jun 09 08:04:02.314550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314424 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jun 09 08:04:02.314550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314506 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250"} Jun 09 08:04:02.314550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314534 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"b5f52318-eb86-4d9c-99b9-75540b99b35f","Type":"ContainerDied","Data":"2d02213eaac1ae3c8c2f2a0e249e7d4238d0823cadc196defd53e67f5d41759f"} Jun 09 08:04:02.314672 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.314572 2573 scope.go:117] "RemoveContainer" containerID="300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" Jun 09 08:04:02.323518 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.323500 2573 scope.go:117] "RemoveContainer" containerID="42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" Jun 09 08:04:02.330699 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.330684 2573 scope.go:117] "RemoveContainer" containerID="a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" Jun 09 08:04:02.336926 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.336911 2573 scope.go:117] "RemoveContainer" containerID="5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" Jun 09 08:04:02.341424 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.341403 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:04:02.344029 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.344002 2573 scope.go:117] "RemoveContainer" containerID="c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" Jun 09 08:04:02.349008 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.348986 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jun 09 08:04:02.351449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.351428 2573 scope.go:117] "RemoveContainer" containerID="434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" Jun 09 08:04:02.358087 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.358067 2573 scope.go:117] "RemoveContainer" containerID="1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2" Jun 09 08:04:02.364515 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.364498 2573 scope.go:117] "RemoveContainer" containerID="300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" Jun 09 08:04:02.364757 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.364738 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939\": container with ID starting with 300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939 not found: ID does not exist" containerID="300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" Jun 09 08:04:02.364816 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.364763 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939"} err="failed to get container status \"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939\": rpc error: code = NotFound desc = could not find container \"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939\": container with ID starting with 300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939 not found: ID does not exist" Jun 09 08:04:02.364816 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.364781 2573 scope.go:117] "RemoveContainer" containerID="42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" Jun 09 08:04:02.365028 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.365012 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0\": container with ID starting with 42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0 not found: ID does not exist" containerID="42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" Jun 09 08:04:02.365065 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365032 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0"} err="failed to get container status \"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0\": rpc error: code = NotFound desc = could not find container \"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0\": container with ID starting with 42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0 not found: ID does not exist" Jun 09 08:04:02.365065 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365045 2573 scope.go:117] "RemoveContainer" containerID="a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" Jun 09 08:04:02.365260 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.365244 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3\": container with ID starting with a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3 not found: ID does not exist" containerID="a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" Jun 09 08:04:02.365311 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365264 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3"} err="failed to get container status \"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3\": rpc error: code = NotFound desc = could not find container \"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3\": container with ID starting with a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3 not found: ID does not exist" Jun 09 08:04:02.365311 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365276 2573 scope.go:117] "RemoveContainer" containerID="5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" Jun 09 08:04:02.365482 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.365466 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250\": container with ID starting with 5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250 not found: ID does not exist" containerID="5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" Jun 09 08:04:02.365520 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365485 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250"} err="failed to get container status \"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250\": rpc error: code = NotFound desc = could not find container \"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250\": container with ID starting with 5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250 not found: ID does not exist" Jun 09 08:04:02.365520 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365497 2573 scope.go:117] "RemoveContainer" containerID="c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" Jun 09 08:04:02.365691 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.365676 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6\": container with ID starting with c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6 not found: ID does not exist" containerID="c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" Jun 09 08:04:02.365729 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365693 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6"} err="failed to get container status \"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6\": rpc error: code = NotFound desc = could not find container \"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6\": container with ID starting with c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6 not found: ID does not exist" Jun 09 08:04:02.365729 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365704 2573 scope.go:117] "RemoveContainer" containerID="434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" Jun 09 08:04:02.365933 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.365910 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0\": container with ID starting with 434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0 not found: ID does not exist" containerID="434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" Jun 09 08:04:02.365981 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365940 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0"} err="failed to get container status \"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0\": rpc error: code = NotFound desc = could not find container \"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0\": container with ID starting with 434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0 not found: ID does not exist" Jun 09 08:04:02.365981 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.365956 2573 scope.go:117] "RemoveContainer" containerID="1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2" Jun 09 08:04:02.366179 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:02.366162 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2\": container with ID starting with 1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2 not found: ID does not exist" containerID="1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2" Jun 09 08:04:02.366225 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366182 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2"} err="failed to get container status \"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2\": rpc error: code = NotFound desc = could not find container \"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2\": container with ID starting with 1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2 not found: ID does not exist" Jun 09 08:04:02.366225 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366197 2573 scope.go:117] "RemoveContainer" containerID="300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939" Jun 09 08:04:02.366380 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366364 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939"} err="failed to get container status \"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939\": rpc error: code = NotFound desc = could not find container \"300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939\": container with ID starting with 300e9e22f1f137b886923927793907d1819803da34431a725744c42060975939 not found: ID does not exist" Jun 09 08:04:02.366380 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366379 2573 scope.go:117] "RemoveContainer" containerID="42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0" Jun 09 08:04:02.366588 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366572 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0"} err="failed to get container status \"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0\": rpc error: code = NotFound desc = could not find container \"42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0\": container with ID starting with 42f536257c0c5a2c8aa4e19c4a82be11c11f4370359bf66b842c669d918c76a0 not found: ID does not exist" Jun 09 08:04:02.366637 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366588 2573 scope.go:117] "RemoveContainer" containerID="a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3" Jun 09 08:04:02.366767 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366751 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3"} err="failed to get container status \"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3\": rpc error: code = NotFound desc = could not find container \"a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3\": container with ID starting with a6a2d78b344eb7a459dc0520bf6ea7aad68ac3a4bbebef1cf169016f7bbad8d3 not found: ID does not exist" Jun 09 08:04:02.366767 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366766 2573 scope.go:117] "RemoveContainer" containerID="5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250" Jun 09 08:04:02.366951 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366936 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250"} err="failed to get container status \"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250\": rpc error: code = NotFound desc = could not find container \"5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250\": container with ID starting with 5a5f77bd7dc9e81fd0ab5b13a4a55adfca69c5ed4100e8a823cdfc7be3b97250 not found: ID does not exist" Jun 09 08:04:02.367001 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.366951 2573 scope.go:117] "RemoveContainer" containerID="c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6" Jun 09 08:04:02.367132 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.367119 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6"} err="failed to get container status \"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6\": rpc error: code = NotFound desc = could not find container \"c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6\": container with ID starting with c60fa61e10565305fdb2d9ac24fe45e5720ad93be8fcbe22ee51557ea7ede5d6 not found: ID does not exist" Jun 09 08:04:02.367173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.367132 2573 scope.go:117] "RemoveContainer" containerID="434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0" Jun 09 08:04:02.367299 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.367283 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0"} err="failed to get container status \"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0\": rpc error: code = NotFound desc = could not find container \"434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0\": container with ID starting with 434374adab5ea488e297aaa09d348db1f9a88dce5b7b3481b4024ec0adb8a9e0 not found: ID does not exist" Jun 09 08:04:02.367340 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.367298 2573 scope.go:117] "RemoveContainer" containerID="1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2" Jun 09 08:04:02.367467 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:02.367452 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2"} err="failed to get container status \"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2\": rpc error: code = NotFound desc = could not find container \"1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2\": container with ID starting with 1387392856d9592badb555ea588b5d00b9684fe570ace36c2152625c3a4143d2 not found: ID does not exist" Jun 09 08:04:03.009660 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:03.009624 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" path="/var/lib/kubelet/pods/b5f52318-eb86-4d9c-99b9-75540b99b35f/volumes" Jun 09 08:04:04.639913 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.639855 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:04.640287 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640171 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="prometheus" containerID="cri-o://819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" gracePeriod=600 Jun 09 08:04:04.640287 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640220 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="thanos-sidecar" containerID="cri-o://407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" gracePeriod=600 Jun 09 08:04:04.640287 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640246 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-thanos" containerID="cri-o://9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" gracePeriod=600 Jun 09 08:04:04.640287 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640263 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy" containerID="cri-o://97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" gracePeriod=600 Jun 09 08:04:04.640529 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640213 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-web" containerID="cri-o://93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" gracePeriod=600 Jun 09 08:04:04.640529 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.640346 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="config-reloader" containerID="cri-o://6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" gracePeriod=600 Jun 09 08:04:04.870825 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:04.870801 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.042912 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.042854 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.042912 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.042915 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.042935 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntfqh\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.042957 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.042974 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043007 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043033 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043060 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043097 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043126 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043151 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043180 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043218 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043242 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043266 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043314 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043342 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043384 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043413 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043432 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.043577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043474 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy\") pod \"374b54b8-ce26-4537-bb88-73b9008f84f8\" (UID: \"374b54b8-ce26-4537-bb88-73b9008f84f8\") " Jun 09 08:04:05.044109 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.043695 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.044848 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.044323 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045034 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045429 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045467 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045606 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045677 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.045970 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh" (OuterVolumeSpecName: "kube-api-access-ntfqh") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "kube-api-access-ntfqh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.046094 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.046347 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.046773 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.046732 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:04:05.047263 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.046916 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.047263 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.046932 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 08:04:05.047390 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.047368 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config" (OuterVolumeSpecName: "config") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.047447 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.047394 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out" (OuterVolumeSpecName: "config-out") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 09 08:04:05.047548 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.047529 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.047909 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.047849 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.048023 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.047995 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.048247 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.048225 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:04:05.056074 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.056049 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config" (OuterVolumeSpecName: "web-config") pod "374b54b8-ce26-4537-bb88-73b9008f84f8" (UID: "374b54b8-ce26-4537-bb88-73b9008f84f8"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:04:05.144645 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144607 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-metrics-client-ca\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144645 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144641 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144658 2573 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-kube-rbac-proxy\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144672 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144685 2573 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-config-out\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144700 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntfqh\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-kube-api-access-ntfqh\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144713 2573 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-grpc-tls\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144728 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144742 2573 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144757 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144772 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-db\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.144803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144799 2573 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-config\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144817 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144829 2573 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144839 2573 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/374b54b8-ce26-4537-bb88-73b9008f84f8-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144849 2573 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-web-config\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144857 2573 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-secret-metrics-client-certs\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144891 2573 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/374b54b8-ce26-4537-bb88-73b9008f84f8-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.145123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.144901 2573 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/374b54b8-ce26-4537-bb88-73b9008f84f8-tls-assets\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327711 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327734 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327743 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327750 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327756 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327761 2573 generic.go:356] "Generic (PLEG): container finished" podID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" exitCode=0 Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327784 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.327793 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327787 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327888 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327954 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.327984 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.328026 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.328071 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} Jun 09 08:04:05.328295 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.328114 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"374b54b8-ce26-4537-bb88-73b9008f84f8","Type":"ContainerDied","Data":"e670cfec41620f2ba44482adf87c46f96c041fe8eb946c9f993201d62bde110c"} Jun 09 08:04:05.335384 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.335273 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.342106 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.342084 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.348235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.348214 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.354281 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.354262 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.359706 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.359680 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:05.361012 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.360997 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.368211 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.368185 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.370790 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.370760 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:05.378019 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.377979 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.378424 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.378388 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.378475 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.378442 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.378513 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.378482 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.378794 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.378759 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.378855 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.378800 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.378921 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.378848 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.379153 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.379136 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.379215 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379159 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.379215 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379175 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.379384 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.379367 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.379424 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379389 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.379424 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379403 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.379606 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.379587 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.379646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379612 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.379646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379626 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.379944 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.379910 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.379990 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.379964 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.380039 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380002 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.380341 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:04:05.380323 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.380395 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380346 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.380395 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380361 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.380582 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380560 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.380582 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380580 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.380813 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380797 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.380887 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.380815 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.381068 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381050 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.381106 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381069 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.381310 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381295 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.381379 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381312 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.381528 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381511 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.381528 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381527 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.381742 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381724 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.381779 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381744 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.382010 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.381991 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.382086 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382011 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.382241 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382224 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.382310 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382241 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.382449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382428 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.382495 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382450 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.382683 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382666 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.382736 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382684 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.382902 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382883 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.382943 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.382903 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.383139 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383112 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.383219 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383139 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.383353 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383337 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.383393 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383354 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.383566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383549 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.383603 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383566 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.383782 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383766 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.383846 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.383782 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.384030 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384003 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.384079 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384031 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.384243 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384223 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.384279 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384243 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.384437 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384422 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.384437 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384436 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.384604 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384588 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.384642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384604 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.384823 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384806 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.384897 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.384824 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.385053 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385034 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.385107 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385052 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.385221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385202 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.385262 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385221 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.385431 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385414 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.385467 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385431 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.385638 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385621 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.385682 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385638 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.385825 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385807 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.385895 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.385826 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.386054 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386037 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.386106 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386054 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.386244 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386228 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.386280 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386244 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.386431 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386417 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.386431 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386430 2573 scope.go:117] "RemoveContainer" containerID="9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c" Jun 09 08:04:05.386669 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386654 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c"} err="failed to get container status \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": rpc error: code = NotFound desc = could not find container \"9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c\": container with ID starting with 9d5bd3518a8998d396e3932e7ef659426e014bd5f9919fce59cf462874f4204c not found: ID does not exist" Jun 09 08:04:05.386669 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386667 2573 scope.go:117] "RemoveContainer" containerID="97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380" Jun 09 08:04:05.386881 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386849 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380"} err="failed to get container status \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": rpc error: code = NotFound desc = could not find container \"97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380\": container with ID starting with 97a5bf80e7898d8b5e1f80444ce5204bec9e0f9ff3e7a926f6ec3cac4acbf380 not found: ID does not exist" Jun 09 08:04:05.386927 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.386878 2573 scope.go:117] "RemoveContainer" containerID="93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a" Jun 09 08:04:05.387112 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387094 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a"} err="failed to get container status \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": rpc error: code = NotFound desc = could not find container \"93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a\": container with ID starting with 93cd1d19f093630a7af9b7f73a489b405d8b3e8b48c4b9a269a8c62dde7fad5a not found: ID does not exist" Jun 09 08:04:05.387112 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387111 2573 scope.go:117] "RemoveContainer" containerID="407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9" Jun 09 08:04:05.387325 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387306 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9"} err="failed to get container status \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": rpc error: code = NotFound desc = could not find container \"407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9\": container with ID starting with 407d0d937ae852c90a2177d19a2af9aeeb8853ff652c47fa7b5d24832a6986e9 not found: ID does not exist" Jun 09 08:04:05.387368 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387325 2573 scope.go:117] "RemoveContainer" containerID="6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031" Jun 09 08:04:05.387526 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387509 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031"} err="failed to get container status \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": rpc error: code = NotFound desc = could not find container \"6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031\": container with ID starting with 6fa42f4dfe39e7b67102cc455ca97af53e552bc7e7755057f585e9e7aba2f031 not found: ID does not exist" Jun 09 08:04:05.387526 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387525 2573 scope.go:117] "RemoveContainer" containerID="819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc" Jun 09 08:04:05.387696 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387682 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc"} err="failed to get container status \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": rpc error: code = NotFound desc = could not find container \"819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc\": container with ID starting with 819de186099c7ffab16e6f1b7e3f4500698cf5cd27a84df66eaee9ccc6e800fc not found: ID does not exist" Jun 09 08:04:05.387736 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387697 2573 scope.go:117] "RemoveContainer" containerID="cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0" Jun 09 08:04:05.387899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.387880 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0"} err="failed to get container status \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": rpc error: code = NotFound desc = could not find container \"cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0\": container with ID starting with cf99dce9d9b1e1b00570f591664f1135664fa7f1b114e503494f66e39f2d30d0 not found: ID does not exist" Jun 09 08:04:05.400842 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.400821 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:05.401126 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401112 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="prometheus" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401131 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="prometheus" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401145 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401151 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401157 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401162 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401167 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy" Jun 09 08:04:05.401173 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401172 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401178 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="init-config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401183 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="init-config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401190 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401194 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401203 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="alertmanager" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401211 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="alertmanager" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401221 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="init-config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401226 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="init-config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401232 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401237 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401243 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-metric" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401248 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-metric" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401255 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401260 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="config-reloader" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401268 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="prom-label-proxy" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401273 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="prom-label-proxy" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401280 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="thanos-sidecar" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401288 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="thanos-sidecar" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401298 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-thanos" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401304 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-thanos" Jun 09 08:04:05.401366 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401365 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-metric" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401376 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="prom-label-proxy" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401385 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="config-reloader" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401390 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="thanos-sidecar" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401394 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-thanos" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401400 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="alertmanager" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401404 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="config-reloader" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401409 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="prometheus" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401414 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401419 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401423 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5f52318-eb86-4d9c-99b9-75540b99b35f" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.401958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.401428 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" containerName="kube-rbac-proxy-web" Jun 09 08:04:05.406490 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.406475 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.408639 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.408620 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 09 08:04:05.408934 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.408914 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 09 08:04:05.409044 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.408936 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cg4b041n15gig\"" Jun 09 08:04:05.409044 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.408999 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 09 08:04:05.409151 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409062 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 09 08:04:05.409151 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409079 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 09 08:04:05.409250 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409159 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 09 08:04:05.409311 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409287 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-qkrcw\"" Jun 09 08:04:05.409643 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409626 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 09 08:04:05.409724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409675 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 09 08:04:05.409724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409674 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 09 08:04:05.409814 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.409748 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 09 08:04:05.410134 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.410119 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 09 08:04:05.410208 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.410119 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 09 08:04:05.410409 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.410395 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 09 08:04:05.412905 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.412888 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 09 08:04:05.416376 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.416358 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 09 08:04:05.424833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.424812 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:05.547617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547579 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547625 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-web-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547650 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547682 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547707 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547724 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.547791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547778 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-config-out\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547798 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547836 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547895 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547932 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548027 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.547983 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548047 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548072 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548091 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548108 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548126 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548148 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zp92\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-kube-api-access-8zp92\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.548192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.548176 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.649474 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649390 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-web-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.649474 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649428 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.649474 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649449 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.649474 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649472 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649488 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649599 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-config-out\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649634 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649660 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649690 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649721 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649749 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649776 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649812 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649840 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649890 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649916 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649946 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.649984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zp92\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-kube-api-access-8zp92\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.650010 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.650054 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650952 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.650310 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650952 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.650331 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.650952 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.650795 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.652724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.651682 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.652724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.652386 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.652724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.652670 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.652724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.652711 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-config-out\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.653017 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.652893 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.653108 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.653081 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.653167 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.653134 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.653234 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.653201 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.654242 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.653715 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-web-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.654242 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.653974 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.654242 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.654159 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.654721 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.654696 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.655109 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.655091 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-config\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.655652 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.655630 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.655982 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.655966 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/78344f2f-58a8-4f5a-bc56-4da6299edc55-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.656343 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.656326 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/78344f2f-58a8-4f5a-bc56-4da6299edc55-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.661164 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.660986 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zp92\" (UniqueName: \"kubernetes.io/projected/78344f2f-58a8-4f5a-bc56-4da6299edc55-kube-api-access-8zp92\") pod \"prometheus-k8s-0\" (UID: \"78344f2f-58a8-4f5a-bc56-4da6299edc55\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.716102 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.716065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:05.847897 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:05.847750 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 09 08:04:05.850457 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:04:05.850427 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78344f2f_58a8_4f5a_bc56_4da6299edc55.slice/crio-544b221ca68238637e65eaaf8b79a2d87ef511c05c83e4885746177e605ac249 WatchSource:0}: Error finding container 544b221ca68238637e65eaaf8b79a2d87ef511c05c83e4885746177e605ac249: Status 404 returned error can't find the container with id 544b221ca68238637e65eaaf8b79a2d87ef511c05c83e4885746177e605ac249 Jun 09 08:04:06.333878 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:06.333832 2573 generic.go:356] "Generic (PLEG): container finished" podID="78344f2f-58a8-4f5a-bc56-4da6299edc55" containerID="746ea3fb8a29bc32ff433dceac92b530291e31ff7d799fd4017b748a81c00ff8" exitCode=0 Jun 09 08:04:06.334025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:06.333924 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerDied","Data":"746ea3fb8a29bc32ff433dceac92b530291e31ff7d799fd4017b748a81c00ff8"} Jun 09 08:04:06.334025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:06.333997 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"544b221ca68238637e65eaaf8b79a2d87ef511c05c83e4885746177e605ac249"} Jun 09 08:04:07.015698 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.015663 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="374b54b8-ce26-4537-bb88-73b9008f84f8" path="/var/lib/kubelet/pods/374b54b8-ce26-4537-bb88-73b9008f84f8/volumes" Jun 09 08:04:07.340399 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340324 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"364128cde87863c3f37f903d4de30771e46d5ce90024cdc1c37b6dff109a5668"} Jun 09 08:04:07.340399 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340382 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"9b825b135cbb83e62e86abcdbb53f505f4f92137c9ca5de0af97ddf80a1c31e1"} Jun 09 08:04:07.340566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340409 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"391bd3712dcf44a66d63bb70268bba15f50319c82156dfa10688b45a27c80ab9"} Jun 09 08:04:07.340566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340433 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"577a54e0f30b7707ba32e6e38bacc0ca1d0089ae3107552a0b61f71f4f750b4d"} Jun 09 08:04:07.340566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340455 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"504d3da26cade8d130dc70519bffc9be39b76e983560df226daa99e021d55a89"} Jun 09 08:04:07.340566 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.340477 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"78344f2f-58a8-4f5a-bc56-4da6299edc55","Type":"ContainerStarted","Data":"48314cd108263515f401c2ef9f15debd7a36cbcc4e1821f35933cdf6423013ed"} Jun 09 08:04:07.370300 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:07.370253 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.351128687 podStartE2EDuration="2.370235956s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.107269ms" isStatefulPod=false podCreationTimestamp="2026-06-09 08:04:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:04:07.368547821 +0000 UTC m=+344.809011799" watchObservedRunningTime="2026-06-09 08:04:07.370235956 +0000 UTC m=+344.810699947" Jun 09 08:04:08.277300 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:08.277223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:08.412646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:08.412615 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:09.000104 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:09.000066 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:09.066370 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:09.066332 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:10.717309 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:10.717232 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:04:20.346101 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:20.346059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:20.403391 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:20.403356 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:20.465107 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:20.465073 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:55.885339 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:55.885300 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:04:55.987235 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:04:55.987181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:05.717290 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:05.717227 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:05:05.732817 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:05.732789 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:05:06.557833 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:06.557804 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 09 08:05:14.500320 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:14.500279 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:14.553233 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:14.553196 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:23.469658 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:23.469620 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:23.570452 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:23.570413 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:26.452217 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:26.452181 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:26.508589 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:26.508545 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:34.629845 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:34.629811 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:34.685770 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:34.685732 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:34.742584 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:34.742549 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:41.255277 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:41.255186 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:41.356102 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:41.356059 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:41.549203 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:41.549121 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:05:47.645416 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:05:47.645377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:29.400368 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:29.400331 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:29.475174 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:29.475139 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:56.128140 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:56.128105 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:56.906426 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:56.906395 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:59.755322 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:59.755284 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:06:59.855081 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:06:59.855045 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:01.232634 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:01.232602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:01.347180 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:01.347145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:01.903434 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:01.903397 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:02.001484 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:02.001447 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:05.400172 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:05.400141 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:05.756251 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:05.756202 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:06.533921 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:06.533858 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:06.588819 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:06.588772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:11.575395 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:11.575313 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:11.629062 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:11.629026 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:12.553629 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:12.553574 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:14.044193 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:14.044149 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:14.139754 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:14.139725 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:19.607041 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:19.606997 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:19.710568 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:19.710530 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:21.748620 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:21.748584 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:33.881245 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.881211 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:33.944749 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.944714 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd"] Jun 09 08:07:33.947792 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.947777 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:33.950272 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950247 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 09 08:07:33.950419 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950251 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 09 08:07:33.950419 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950321 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 09 08:07:33.950732 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950714 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-lq9bn\"" Jun 09 08:07:33.950839 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950824 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 09 08:07:33.950971 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.950958 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 09 08:07:33.956685 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.956662 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd"] Jun 09 08:07:33.963072 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:33.963047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:34.021944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.021895 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5vx8\" (UniqueName: \"kubernetes.io/projected/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-kube-api-access-v5vx8\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.022120 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.022016 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-manager-config\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.022120 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.022067 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-metrics-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.022120 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.022095 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.122857 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.122815 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.123039 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.122880 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5vx8\" (UniqueName: \"kubernetes.io/projected/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-kube-api-access-v5vx8\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.123039 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.123028 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-manager-config\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.123163 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.123107 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-metrics-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.123726 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.123698 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-manager-config\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.125408 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.125377 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.125514 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.125433 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-metrics-cert\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.134297 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.134236 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5vx8\" (UniqueName: \"kubernetes.io/projected/9ecf62de-2a78-4782-bfa7-db90d1ad3e77-kube-api-access-v5vx8\") pod \"lws-controller-manager-54ddd4477f-4d4xd\" (UID: \"9ecf62de-2a78-4782-bfa7-db90d1ad3e77\") " pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.257412 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.257359 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:34.386931 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.386727 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd"] Jun 09 08:07:34.389491 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:07:34.389468 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ecf62de_2a78_4782_bfa7_db90d1ad3e77.slice/crio-cb163ee09ea75f1e46d3ba001a32f2bf57ed30c2cd46b7b0ea31b1dbb635424a WatchSource:0}: Error finding container cb163ee09ea75f1e46d3ba001a32f2bf57ed30c2cd46b7b0ea31b1dbb635424a: Status 404 returned error can't find the container with id cb163ee09ea75f1e46d3ba001a32f2bf57ed30c2cd46b7b0ea31b1dbb635424a Jun 09 08:07:34.391367 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:34.391350 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 08:07:35.034556 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:35.034515 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" event={"ID":"9ecf62de-2a78-4782-bfa7-db90d1ad3e77","Type":"ContainerStarted","Data":"cb163ee09ea75f1e46d3ba001a32f2bf57ed30c2cd46b7b0ea31b1dbb635424a"} Jun 09 08:07:37.961471 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:37.961438 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.052028 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.051990 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" event={"ID":"9ecf62de-2a78-4782-bfa7-db90d1ad3e77","Type":"ContainerStarted","Data":"2f60f1e15bbbcee61d6e9ab65c35d351bdee97d14ee678abae097d1dd9fe5164"} Jun 09 08:07:38.052183 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.052099 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:38.070461 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.070427 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.082025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.081973 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" podStartSLOduration=2.458036703 podStartE2EDuration="5.081955389s" totalImagesPullingTime="2.623918686s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:07:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:07:38.081609081 +0000 UTC m=+555.522073060" watchObservedRunningTime="2026-06-09 08:07:38.081955389 +0000 UTC m=+555.522419368" Jun 09 08:07:38.357567 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.357485 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.570208 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.570175 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.811530 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.811488 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.912501 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.912471 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:38.978790 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.978757 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m"] Jun 09 08:07:38.981978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.981957 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:38.984627 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.984603 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 09 08:07:38.985582 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.985565 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 09 08:07:38.985672 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.985580 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 09 08:07:38.986792 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.986773 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-8qlhg\"" Jun 09 08:07:38.989920 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:38.989900 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 09 08:07:39.001977 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.001951 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m"] Jun 09 08:07:39.065095 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.064940 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.065095 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.065001 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-manifests\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.065095 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.065032 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nz44\" (UniqueName: \"kubernetes.io/projected/f4d60053-760c-4d10-b0aa-f08e930c8820-kube-api-access-8nz44\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.065095 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.065055 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.065095 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.065078 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-tmp\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166013 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.165969 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-tmp\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166227 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.166195 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166371 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.166258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-manifests\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166371 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.166303 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nz44\" (UniqueName: \"kubernetes.io/projected/f4d60053-760c-4d10-b0aa-f08e930c8820-kube-api-access-8nz44\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166371 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.166330 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.166666 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.166639 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-manifests\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.168345 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.168327 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f4d60053-760c-4d10-b0aa-f08e930c8820-tmp\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.168610 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.168591 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-apiservice-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.168676 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.168594 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f4d60053-760c-4d10-b0aa-f08e930c8820-webhook-cert\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.183379 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.183358 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nz44\" (UniqueName: \"kubernetes.io/projected/f4d60053-760c-4d10-b0aa-f08e930c8820-kube-api-access-8nz44\") pod \"opendatahub-operator-controller-manager-c6b94fd7b-s497m\" (UID: \"f4d60053-760c-4d10-b0aa-f08e930c8820\") " pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.204798 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.204764 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:39.292611 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.292574 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:39.305605 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.305560 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:39.405377 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.405332 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:39.434835 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.434801 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m"] Jun 09 08:07:39.438284 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:07:39.438260 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4d60053_760c_4d10_b0aa_f08e930c8820.slice/crio-a012aa2cba3d40be8800074d1756f80ed7ad4718e207cd1faafd10a8ed798861 WatchSource:0}: Error finding container a012aa2cba3d40be8800074d1756f80ed7ad4718e207cd1faafd10a8ed798861: Status 404 returned error can't find the container with id a012aa2cba3d40be8800074d1756f80ed7ad4718e207cd1faafd10a8ed798861 Jun 09 08:07:39.509908 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.509847 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:39.610679 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:39.610574 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:40.063479 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:40.063435 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" event={"ID":"f4d60053-760c-4d10-b0aa-f08e930c8820","Type":"ContainerStarted","Data":"a012aa2cba3d40be8800074d1756f80ed7ad4718e207cd1faafd10a8ed798861"} Jun 09 08:07:43.081001 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:43.080959 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" event={"ID":"f4d60053-760c-4d10-b0aa-f08e930c8820","Type":"ContainerStarted","Data":"d07b41fabe02f5ce4f3f35280bcbed1d6252dedb5fb0d75ca06e1c5a9b3039ec"} Jun 09 08:07:44.086123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:44.086091 2573 generic.go:356] "Generic (PLEG): container finished" podID="f4d60053-760c-4d10-b0aa-f08e930c8820" containerID="d07b41fabe02f5ce4f3f35280bcbed1d6252dedb5fb0d75ca06e1c5a9b3039ec" exitCode=0 Jun 09 08:07:44.086552 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:44.086190 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" event={"ID":"f4d60053-760c-4d10-b0aa-f08e930c8820","Type":"ContainerDied","Data":"d07b41fabe02f5ce4f3f35280bcbed1d6252dedb5fb0d75ca06e1c5a9b3039ec"} Jun 09 08:07:45.092821 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:45.092782 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" event={"ID":"f4d60053-760c-4d10-b0aa-f08e930c8820","Type":"ContainerStarted","Data":"2477fa06bd70fcd70d6597d5f6e7b060427c9b732712829d4e58250bc1638151"} Jun 09 08:07:45.093228 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:45.092917 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:45.114959 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:45.114896 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" podStartSLOduration=2.984075877 podStartE2EDuration="7.114877831s" totalImagesPullingTime="3.132684216s" totalInitContainerRuntime="998.117738ms" isStatefulPod=false podCreationTimestamp="2026-06-09 08:07:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:07:45.114489545 +0000 UTC m=+562.554953550" watchObservedRunningTime="2026-06-09 08:07:45.114877831 +0000 UTC m=+562.555341800" Jun 09 08:07:49.058818 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:49.058781 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-54ddd4477f-4d4xd" Jun 09 08:07:50.178478 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:50.178430 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:50.278058 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:50.278017 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:56.099217 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:56.099190 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-c6b94fd7b-s497m" Jun 09 08:07:57.970170 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:57.970135 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:58.373514 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.373436 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-69895d85c6-x79h4"] Jun 09 08:07:58.384338 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.384302 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.386356 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.386325 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 09 08:07:58.387133 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.387114 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 09 08:07:58.387243 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.387130 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-ht4ld\"" Jun 09 08:07:58.387243 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.387121 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 09 08:07:58.387243 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.387192 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 09 08:07:58.390637 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.390608 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69895d85c6-x79h4"] Jun 09 08:07:58.525826 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.525788 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxps4\" (UniqueName: \"kubernetes.io/projected/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-kube-api-access-mxps4\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.526022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.525846 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tmp\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.526022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.525914 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tls-certs\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.627211 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.627112 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxps4\" (UniqueName: \"kubernetes.io/projected/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-kube-api-access-mxps4\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.627211 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.627176 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tmp\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.627211 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.627200 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tls-certs\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.629420 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.629399 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tmp\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.629637 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.629618 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-tls-certs\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.641218 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.641193 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxps4\" (UniqueName: \"kubernetes.io/projected/3bafe8a0-1173-4ad3-afc2-054f00a90e7b-kube-api-access-mxps4\") pod \"kube-auth-proxy-69895d85c6-x79h4\" (UID: \"3bafe8a0-1173-4ad3-afc2-054f00a90e7b\") " pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.695248 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.695215 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" Jun 09 08:07:58.830530 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:58.830488 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-69895d85c6-x79h4"] Jun 09 08:07:58.833400 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:07:58.833372 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bafe8a0_1173_4ad3_afc2_054f00a90e7b.slice/crio-896e5db7cd026fe59ce0a1fee2f2abb72838178b9aa49e96e4a33c0fc8faf556 WatchSource:0}: Error finding container 896e5db7cd026fe59ce0a1fee2f2abb72838178b9aa49e96e4a33c0fc8faf556: Status 404 returned error can't find the container with id 896e5db7cd026fe59ce0a1fee2f2abb72838178b9aa49e96e4a33c0fc8faf556 Jun 09 08:07:59.049760 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:59.049729 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:59.145265 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:59.145231 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" event={"ID":"3bafe8a0-1173-4ad3-afc2-054f00a90e7b","Type":"ContainerStarted","Data":"896e5db7cd026fe59ce0a1fee2f2abb72838178b9aa49e96e4a33c0fc8faf556"} Jun 09 08:07:59.199065 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:59.199029 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:59.291792 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:59.291747 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:07:59.387901 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:07:59.387793 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:03.162852 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:03.162810 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" event={"ID":"3bafe8a0-1173-4ad3-afc2-054f00a90e7b","Type":"ContainerStarted","Data":"1e9b3367a467b526ff3c74d0c67fcc4da80a139408200addabd72b91a2b92293"} Jun 09 08:08:03.178804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:03.178749 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-69895d85c6-x79h4" podStartSLOduration=0.935721396 podStartE2EDuration="5.178731247s" totalImagesPullingTime="4.243009851s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:07:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:08:03.176957174 +0000 UTC m=+580.617421152" watchObservedRunningTime="2026-06-09 08:08:03.178731247 +0000 UTC m=+580.619195224" Jun 09 08:08:08.545092 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:08.545061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:08.719080 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:08.719047 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:08.917449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:08.917361 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:09.002053 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:09.002012 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:09.226042 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:09.226001 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:09.318618 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:09.318579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:09.922462 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:09.922421 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:10.019131 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:10.019087 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:11.355317 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:11.355282 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:11.424144 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:11.424110 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:15.742381 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:15.742345 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:15.842938 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:15.842904 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:17.350030 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:17.349987 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:17.463725 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:17.463685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:17.717769 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:17.717724 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:18.270695 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:18.270653 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:18.378638 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:18.378602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:28.688899 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:28.688834 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:45.630631 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:45.630593 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:47.637014 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:47.636971 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:47.698701 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:47.698669 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:59.676840 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:59.676801 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:59.737358 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:59.737324 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:08:59.841449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:08:59.841399 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:03.498303 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:03.498269 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:03.556503 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:03.556468 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:04.016692 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:04.016657 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:06.759159 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:06.759122 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:06.859049 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:06.859019 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:07.060453 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:07.060377 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:07.217932 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:07.217895 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:07.319837 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:07.319741 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:07.789192 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:07.789155 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:09.468446 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:09.468406 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:09.528895 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:09.528844 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:20.890023 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:20.889986 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:21.021165 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:21.021127 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:21.071318 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:21.071283 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:28.004306 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:28.004259 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:28.606703 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:28.606670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:29.671617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:29.671579 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:33.226724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:33.226686 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:35.832427 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:35.832384 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:36.184976 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:36.184946 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:38.294940 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:38.294903 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:38.424365 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:38.424328 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:40.877342 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:40.877304 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:43.446675 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:43.446638 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:43.551160 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:43.551125 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:43.651670 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:43.651634 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:43.742649 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:43.742559 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:47.254288 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:47.254259 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:47.341556 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:47.341524 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:50.062565 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:50.062522 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:50.163769 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:50.163731 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:51.421471 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:51.421430 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:51.523779 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:51.523747 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:53.373972 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.373943 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dd968"] Jun 09 08:09:53.377011 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.376993 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.379120 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.379092 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 09 08:09:53.379312 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.379152 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 09 08:09:53.379455 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.379440 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-7vhhw\"" Jun 09 08:09:53.379729 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.379709 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 09 08:09:53.379828 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.379816 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 09 08:09:53.386317 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.386298 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dd968"] Jun 09 08:09:53.390999 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.390971 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:53.537224 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.537188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.537393 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.537243 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxmlg\" (UniqueName: \"kubernetes.io/projected/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-kube-api-access-sxmlg\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.537393 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.537331 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.638214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.638124 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxmlg\" (UniqueName: \"kubernetes.io/projected/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-kube-api-access-sxmlg\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.638214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.638180 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.638401 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.638218 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.638801 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.638781 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.640787 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.640763 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.644441 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.644417 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:53.646200 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.646178 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxmlg\" (UniqueName: \"kubernetes.io/projected/c2dc76d0-4c62-4cf1-b319-13af33c9fa4b-kube-api-access-sxmlg\") pod \"kuadrant-console-plugin-66b5774985-dd968\" (UID: \"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.686028 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.686006 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" Jun 09 08:09:53.814624 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:53.814598 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-dd968"] Jun 09 08:09:53.817149 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:09:53.817116 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc2dc76d0_4c62_4cf1_b319_13af33c9fa4b.slice/crio-722a3cdcf33e8cedf52c85659f4e9ef839d949bafd38fb4d32e97f67c741ae82 WatchSource:0}: Error finding container 722a3cdcf33e8cedf52c85659f4e9ef839d949bafd38fb4d32e97f67c741ae82: Status 404 returned error can't find the container with id 722a3cdcf33e8cedf52c85659f4e9ef839d949bafd38fb4d32e97f67c741ae82 Jun 09 08:09:54.559631 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:54.559595 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" event={"ID":"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b","Type":"ContainerStarted","Data":"722a3cdcf33e8cedf52c85659f4e9ef839d949bafd38fb4d32e97f67c741ae82"} Jun 09 08:09:55.038538 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:55.038499 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:09:55.131296 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:09:55.131257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:02.889258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:02.889215 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:02.987938 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:02.987836 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:03.372805 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:03.372772 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:03.520788 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:03.520756 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:05.896130 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:05.896089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:06.000803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:06.000752 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:11.056840 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:11.056744 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:11.155815 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:11.155766 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:14.455546 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:14.455459 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:14.556472 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:14.556431 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:20.669169 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:20.669129 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" event={"ID":"c2dc76d0-4c62-4cf1-b319-13af33c9fa4b","Type":"ContainerStarted","Data":"6bdb8a6bc46b1774c0ff98d77d04e4e5f3c61f6bbeb9df6f7fe450f5b5e171c2"} Jun 09 08:10:20.692421 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:20.692361 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-dd968" podStartSLOduration=1.376372302 podStartE2EDuration="27.692344213s" totalImagesPullingTime="26.315971911s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:09:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:10:20.6919739 +0000 UTC m=+718.132437879" watchObservedRunningTime="2026-06-09 08:10:20.692344213 +0000 UTC m=+718.132808190" Jun 09 08:10:33.107311 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:33.107276 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:33.169030 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:33.168992 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:36.802281 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:36.802241 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:39.028056 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:39.028011 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:39.082670 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:39.082629 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:47.256004 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.255951 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:47.256473 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.256088 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:10:47.327639 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.327600 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:10:47.327841 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.327737 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.329843 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.329819 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 09 08:10:47.346334 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.346298 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:47.357093 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.357055 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:10:47.400689 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.400656 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:47.421093 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.421056 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.421276 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.421188 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2t7w\" (UniqueName: \"kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.522642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.522546 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2t7w\" (UniqueName: \"kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.522642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.522589 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.523258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.523238 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.532124 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.532103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2t7w\" (UniqueName: \"kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w\") pod \"limitador-limitador-64c94fd64f-jrjbf\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.640469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.640433 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:47.772785 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:47.772707 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:10:47.775627 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:10:47.775589 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20fac7fe_105d_430f_a945_dd30e3f9c79a.slice/crio-4b8f72898dfc0da7efc0d709fa0092450062646fc91ee197cd9985d51deb0353 WatchSource:0}: Error finding container 4b8f72898dfc0da7efc0d709fa0092450062646fc91ee197cd9985d51deb0353: Status 404 returned error can't find the container with id 4b8f72898dfc0da7efc0d709fa0092450062646fc91ee197cd9985d51deb0353 Jun 09 08:10:48.179969 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.179926 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:48.264653 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.264619 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:10:48.265661 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.265636 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:10:48.267746 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.267728 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:10:48.269731 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.269708 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hshs9\"" Jun 09 08:10:48.273495 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.273473 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:10:48.430947 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.430844 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkkmf\" (UniqueName: \"kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf\") pod \"authorino-75d556bd77-qhstm\" (UID: \"1d9138c7-a857-4b1c-929d-0ea3d680cdde\") " pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:10:48.531665 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.531625 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkkmf\" (UniqueName: \"kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf\") pod \"authorino-75d556bd77-qhstm\" (UID: \"1d9138c7-a857-4b1c-929d-0ea3d680cdde\") " pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:10:48.541831 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.541791 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkkmf\" (UniqueName: \"kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf\") pod \"authorino-75d556bd77-qhstm\" (UID: \"1d9138c7-a857-4b1c-929d-0ea3d680cdde\") " pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:10:48.579440 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.579396 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:10:48.756353 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.756283 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:10:48.762975 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:10:48.762933 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d9138c7_a857_4b1c_929d_0ea3d680cdde.slice/crio-4f69d3dfcd925f324a51d26153ffc11216820b7541f27235c4048f63fbf26787 WatchSource:0}: Error finding container 4f69d3dfcd925f324a51d26153ffc11216820b7541f27235c4048f63fbf26787: Status 404 returned error can't find the container with id 4f69d3dfcd925f324a51d26153ffc11216820b7541f27235c4048f63fbf26787 Jun 09 08:10:48.773745 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.773706 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" event={"ID":"20fac7fe-105d-430f-a945-dd30e3f9c79a","Type":"ContainerStarted","Data":"4b8f72898dfc0da7efc0d709fa0092450062646fc91ee197cd9985d51deb0353"} Jun 09 08:10:48.778090 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:48.778041 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qhstm" event={"ID":"1d9138c7-a857-4b1c-929d-0ea3d680cdde","Type":"ContainerStarted","Data":"4f69d3dfcd925f324a51d26153ffc11216820b7541f27235c4048f63fbf26787"} Jun 09 08:10:51.794001 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:51.793960 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" event={"ID":"20fac7fe-105d-430f-a945-dd30e3f9c79a","Type":"ContainerStarted","Data":"3fd2ae5fbd4c7d8ae285504439f46290b22c33cdea251f6ee484db9d059d5868"} Jun 09 08:10:51.794532 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:51.794066 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:10:51.811775 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:51.811723 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" podStartSLOduration=1.7880703580000001 podStartE2EDuration="4.811706226s" totalImagesPullingTime="3.023635868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:10:51.809957719 +0000 UTC m=+749.250421699" watchObservedRunningTime="2026-06-09 08:10:51.811706226 +0000 UTC m=+749.252170225" Jun 09 08:10:52.799727 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:52.799686 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qhstm" event={"ID":"1d9138c7-a857-4b1c-929d-0ea3d680cdde","Type":"ContainerStarted","Data":"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451"} Jun 09 08:10:52.814411 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:10:52.814344 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-qhstm" podStartSLOduration=0.886089599 podStartE2EDuration="4.814322103s" totalImagesPullingTime="3.928232504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:10:52.813585318 +0000 UTC m=+750.254049333" watchObservedRunningTime="2026-06-09 08:10:52.814322103 +0000 UTC m=+750.254786082" Jun 09 08:11:02.300313 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:02.300274 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:11:02.300710 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:02.300474 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" podUID="20fac7fe-105d-430f-a945-dd30e3f9c79a" containerName="limitador" containerID="cri-o://3fd2ae5fbd4c7d8ae285504439f46290b22c33cdea251f6ee484db9d059d5868" gracePeriod=30 Jun 09 08:11:02.301197 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:02.301179 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:11:02.840097 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:02.840057 2573 generic.go:356] "Generic (PLEG): container finished" podID="20fac7fe-105d-430f-a945-dd30e3f9c79a" containerID="3fd2ae5fbd4c7d8ae285504439f46290b22c33cdea251f6ee484db9d059d5868" exitCode=0 Jun 09 08:11:02.840245 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:02.840133 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" event={"ID":"20fac7fe-105d-430f-a945-dd30e3f9c79a","Type":"ContainerDied","Data":"3fd2ae5fbd4c7d8ae285504439f46290b22c33cdea251f6ee484db9d059d5868"} Jun 09 08:11:03.257644 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.257612 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:11:03.356156 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.356113 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file\") pod \"20fac7fe-105d-430f-a945-dd30e3f9c79a\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " Jun 09 08:11:03.356549 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.356223 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2t7w\" (UniqueName: \"kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w\") pod \"20fac7fe-105d-430f-a945-dd30e3f9c79a\" (UID: \"20fac7fe-105d-430f-a945-dd30e3f9c79a\") " Jun 09 08:11:03.356549 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.356454 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file" (OuterVolumeSpecName: "config-file") pod "20fac7fe-105d-430f-a945-dd30e3f9c79a" (UID: "20fac7fe-105d-430f-a945-dd30e3f9c79a"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 09 08:11:03.358449 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.358423 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w" (OuterVolumeSpecName: "kube-api-access-n2t7w") pod "20fac7fe-105d-430f-a945-dd30e3f9c79a" (UID: "20fac7fe-105d-430f-a945-dd30e3f9c79a"). InnerVolumeSpecName "kube-api-access-n2t7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:11:03.457493 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.457447 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2t7w\" (UniqueName: \"kubernetes.io/projected/20fac7fe-105d-430f-a945-dd30e3f9c79a-kube-api-access-n2t7w\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:11:03.457493 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.457487 2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/20fac7fe-105d-430f-a945-dd30e3f9c79a-config-file\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:11:03.487064 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.487022 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-9cprz"] Jun 09 08:11:03.487455 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.487434 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20fac7fe-105d-430f-a945-dd30e3f9c79a" containerName="limitador" Jun 09 08:11:03.487455 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.487456 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="20fac7fe-105d-430f-a945-dd30e3f9c79a" containerName="limitador" Jun 09 08:11:03.487617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.487550 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="20fac7fe-105d-430f-a945-dd30e3f9c79a" containerName="limitador" Jun 09 08:11:03.490605 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.490581 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.492473 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.492454 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 09 08:11:03.492584 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.492526 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zmpcp\"" Jun 09 08:11:03.500151 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.500110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-9cprz"] Jun 09 08:11:03.519270 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.519233 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:11:03.558107 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.558073 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2xh2\" (UniqueName: \"kubernetes.io/projected/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-kube-api-access-c2xh2\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.558258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.558112 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-data\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.659525 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.659481 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2xh2\" (UniqueName: \"kubernetes.io/projected/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-kube-api-access-c2xh2\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.659714 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.659534 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-data\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.660022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.660000 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-data\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.668207 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.668184 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2xh2\" (UniqueName: \"kubernetes.io/projected/5f5b57c4-23fe-4ef1-b3ae-145efe8025cc-kube-api-access-c2xh2\") pod \"postgres-69fb4594cf-9cprz\" (UID: \"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc\") " pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.802483 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.802391 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:03.846661 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.846634 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" Jun 09 08:11:03.846661 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.846655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-jrjbf" event={"ID":"20fac7fe-105d-430f-a945-dd30e3f9c79a","Type":"ContainerDied","Data":"4b8f72898dfc0da7efc0d709fa0092450062646fc91ee197cd9985d51deb0353"} Jun 09 08:11:03.846901 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.846716 2573 scope.go:117] "RemoveContainer" containerID="3fd2ae5fbd4c7d8ae285504439f46290b22c33cdea251f6ee484db9d059d5868" Jun 09 08:11:03.869766 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.869733 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:11:03.872977 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.872950 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-jrjbf"] Jun 09 08:11:03.941363 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:03.941276 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-9cprz"] Jun 09 08:11:03.945277 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:11:03.945241 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f5b57c4_23fe_4ef1_b3ae_145efe8025cc.slice/crio-3dc391bbaf66c5cd66893868212d7da5987ce55209963b690ba7a266aef21fe6 WatchSource:0}: Error finding container 3dc391bbaf66c5cd66893868212d7da5987ce55209963b690ba7a266aef21fe6: Status 404 returned error can't find the container with id 3dc391bbaf66c5cd66893868212d7da5987ce55209963b690ba7a266aef21fe6 Jun 09 08:11:04.857737 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:04.853306 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-9cprz" event={"ID":"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc","Type":"ContainerStarted","Data":"3dc391bbaf66c5cd66893868212d7da5987ce55209963b690ba7a266aef21fe6"} Jun 09 08:11:05.013074 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:05.013040 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20fac7fe-105d-430f-a945-dd30e3f9c79a" path="/var/lib/kubelet/pods/20fac7fe-105d-430f-a945-dd30e3f9c79a/volumes" Jun 09 08:11:06.864093 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:06.863999 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-9cprz" event={"ID":"5f5b57c4-23fe-4ef1-b3ae-145efe8025cc","Type":"ContainerStarted","Data":"453fc45765b290fcb44f0a937d7a94a0cc03ef32594a2fe17118b764ddcf415b"} Jun 09 08:11:06.864461 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:06.864132 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:06.879603 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:06.879552 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-9cprz" podStartSLOduration=1.325275797 podStartE2EDuration="3.879534869s" totalImagesPullingTime="2.554259072s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:11:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:11:06.878279152 +0000 UTC m=+764.318743130" watchObservedRunningTime="2026-06-09 08:11:06.879534869 +0000 UTC m=+764.319998877" Jun 09 08:11:12.901779 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:12.901747 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-9cprz" Jun 09 08:11:13.209984 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:13.209937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:11:13.305585 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:13.305544 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:11:13.440262 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:13.440220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:11:38.595314 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:38.595279 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:11:38.999725 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:38.999690 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:11:39.009117 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.009085 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:39.011351 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.011322 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 08:11:39.011520 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.011421 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-g76b9\"" Jun 09 08:11:39.011807 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.011789 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 08:11:39.011949 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.011907 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 08:11:39.014647 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.014625 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:11:39.167458 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.167418 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6h2xv\" (UniqueName: \"kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv\") pod \"maas-keycloak-0\" (UID: \"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:39.268448 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.268352 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6h2xv\" (UniqueName: \"kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv\") pod \"maas-keycloak-0\" (UID: \"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:39.279506 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.279471 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6h2xv\" (UniqueName: \"kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv\") pod \"maas-keycloak-0\" (UID: \"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:39.320222 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.320193 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:39.445233 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.445198 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:11:39.448690 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:11:39.448663 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod633c3b77_c78f_4eb7_8bdf_8c8783beb1cb.slice/crio-64f1d2c0acb91096f730e51ac7fc4baa0bf4e79a13f606525ee78ed3484ee877 WatchSource:0}: Error finding container 64f1d2c0acb91096f730e51ac7fc4baa0bf4e79a13f606525ee78ed3484ee877: Status 404 returned error can't find the container with id 64f1d2c0acb91096f730e51ac7fc4baa0bf4e79a13f606525ee78ed3484ee877 Jun 09 08:11:39.997594 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:39.997556 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb","Type":"ContainerStarted","Data":"64f1d2c0acb91096f730e51ac7fc4baa0bf4e79a13f606525ee78ed3484ee877"} Jun 09 08:11:46.024786 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:46.024748 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb","Type":"ContainerStarted","Data":"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648"} Jun 09 08:11:46.046884 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:46.045377 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.6098077960000001 podStartE2EDuration="8.045352383s" totalImagesPullingTime="6.435544587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:11:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:11:46.041006021 +0000 UTC m=+803.481469998" watchObservedRunningTime="2026-06-09 08:11:46.045352383 +0000 UTC m=+803.485816363" Jun 09 08:11:46.321340 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:46.321300 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:46.322958 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:46.322923 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:47.321662 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:47.321616 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:48.321229 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:48.321159 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:49.320357 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:49.320310 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:49.321010 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:49.320966 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:50.321091 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:50.321042 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:51.321585 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:51.321537 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:52.320740 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:52.320682 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:53.321204 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:53.321149 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:54.320978 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:54.320938 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:55.320927 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:55.320849 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:56.321430 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:56.321378 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:57.321194 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:57.321137 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:58.321420 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:58.321378 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.22:9000/health/started\": dial tcp 10.133.0.22:9000: connect: connection refused" Jun 09 08:11:59.435055 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:59.435011 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 08:11:59.448516 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:11:59.448473 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 08:12:01.871181 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:01.871148 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:05.113709 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:05.113675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:09.441035 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:09.440996 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 08:12:10.090055 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:10.090019 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:10.194828 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:10.194792 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:12.268008 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.267974 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:12:12.268442 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.268192 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-qhstm" podUID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" containerName="authorino" containerID="cri-o://1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451" gracePeriod=30 Jun 09 08:12:12.500436 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.500411 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:12:12.567500 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.567411 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mkkmf\" (UniqueName: \"kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf\") pod \"1d9138c7-a857-4b1c-929d-0ea3d680cdde\" (UID: \"1d9138c7-a857-4b1c-929d-0ea3d680cdde\") " Jun 09 08:12:12.569468 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.569443 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf" (OuterVolumeSpecName: "kube-api-access-mkkmf") pod "1d9138c7-a857-4b1c-929d-0ea3d680cdde" (UID: "1d9138c7-a857-4b1c-929d-0ea3d680cdde"). InnerVolumeSpecName "kube-api-access-mkkmf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:12:12.668010 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:12.667959 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mkkmf\" (UniqueName: \"kubernetes.io/projected/1d9138c7-a857-4b1c-929d-0ea3d680cdde-kube-api-access-mkkmf\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:12:13.153977 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.153927 2573 generic.go:356] "Generic (PLEG): container finished" podID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" containerID="1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451" exitCode=0 Jun 09 08:12:13.153977 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.153971 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-qhstm" Jun 09 08:12:13.154194 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.154007 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qhstm" event={"ID":"1d9138c7-a857-4b1c-929d-0ea3d680cdde","Type":"ContainerDied","Data":"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451"} Jun 09 08:12:13.154194 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.154044 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-qhstm" event={"ID":"1d9138c7-a857-4b1c-929d-0ea3d680cdde","Type":"ContainerDied","Data":"4f69d3dfcd925f324a51d26153ffc11216820b7541f27235c4048f63fbf26787"} Jun 09 08:12:13.154194 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.154064 2573 scope.go:117] "RemoveContainer" containerID="1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451" Jun 09 08:12:13.162114 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.162095 2573 scope.go:117] "RemoveContainer" containerID="1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451" Jun 09 08:12:13.162354 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:12:13.162339 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451\": container with ID starting with 1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451 not found: ID does not exist" containerID="1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451" Jun 09 08:12:13.162400 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.162361 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451"} err="failed to get container status \"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451\": rpc error: code = NotFound desc = could not find container \"1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451\": container with ID starting with 1e885fe736b5d8857875416cb49e4e0434c518e0d62f860344040d2f29f03451 not found: ID does not exist" Jun 09 08:12:13.169559 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.169536 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:12:13.173353 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:13.173325 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-qhstm"] Jun 09 08:12:15.011000 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.010965 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" path="/var/lib/kubelet/pods/1d9138c7-a857-4b1c-929d-0ea3d680cdde/volumes" Jun 09 08:12:15.730148 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.730105 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:15.742515 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.742484 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:15.742759 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.742747 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" containerName="authorino" Jun 09 08:12:15.742804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.742761 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" containerName="authorino" Jun 09 08:12:15.742838 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.742824 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d9138c7-a857-4b1c-929d-0ea3d680cdde" containerName="authorino" Jun 09 08:12:15.745283 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.745268 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:15.747247 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.747218 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 08:12:15.747247 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.747235 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 08:12:15.747247 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.747225 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zxkcm\"" Jun 09 08:12:15.753834 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.753799 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:15.826026 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.825989 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:15.890795 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.890758 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqn2c\" (UniqueName: \"kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:15.890986 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.890818 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:15.920522 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.920491 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:15.923809 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.923779 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:12:15.925903 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.925884 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:15.934570 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.934547 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:12:15.992175 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.992093 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqn2c\" (UniqueName: \"kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:15.992454 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.992426 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:15.994804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:15.994785 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:16.005973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.005948 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqn2c\" (UniqueName: \"kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c\") pod \"maas-controller-676b4f5957-cwzpj\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:16.014478 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.014455 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:16.055609 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.055582 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:16.093682 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.093646 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgr4p\" (UniqueName: \"kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.093845 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.093696 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.180401 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.180372 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:16.183795 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:12:16.183772 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9496e217_5e6b_45eb_9bf0_55c9ddcf0958.slice/crio-bfed85677ddbdc95213e0a3b1a3459808a730f47dc5cfbcd26aad3f4717b7932 WatchSource:0}: Error finding container bfed85677ddbdc95213e0a3b1a3459808a730f47dc5cfbcd26aad3f4717b7932: Status 404 returned error can't find the container with id bfed85677ddbdc95213e0a3b1a3459808a730f47dc5cfbcd26aad3f4717b7932 Jun 09 08:12:16.194840 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.194814 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgr4p\" (UniqueName: \"kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.194938 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.194847 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.197203 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.197184 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.202405 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.202383 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgr4p\" (UniqueName: \"kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p\") pod \"maas-controller-7f796d5f6-lfc85\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.235426 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.235385 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:16.358361 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:16.358337 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:12:16.360683 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:12:16.360651 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ae0ebf7_19ce_48e2_88e2_c10127bd28f3.slice/crio-4f0943f0b8f56628b83b27cecba18fb319897eada08243d2361d507711a2eaca WatchSource:0}: Error finding container 4f0943f0b8f56628b83b27cecba18fb319897eada08243d2361d507711a2eaca: Status 404 returned error can't find the container with id 4f0943f0b8f56628b83b27cecba18fb319897eada08243d2361d507711a2eaca Jun 09 08:12:17.177877 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:17.177822 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-676b4f5957-cwzpj" event={"ID":"9496e217-5e6b-45eb-9bf0-55c9ddcf0958","Type":"ContainerStarted","Data":"bfed85677ddbdc95213e0a3b1a3459808a730f47dc5cfbcd26aad3f4717b7932"} Jun 09 08:12:17.179339 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:17.179308 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-lfc85" event={"ID":"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3","Type":"ContainerStarted","Data":"4f0943f0b8f56628b83b27cecba18fb319897eada08243d2361d507711a2eaca"} Jun 09 08:12:19.149371 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.149338 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:19.195896 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.195839 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-676b4f5957-cwzpj" event={"ID":"9496e217-5e6b-45eb-9bf0-55c9ddcf0958","Type":"ContainerStarted","Data":"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68"} Jun 09 08:12:19.196081 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.195950 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:19.197323 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.197301 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-lfc85" event={"ID":"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3","Type":"ContainerStarted","Data":"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77"} Jun 09 08:12:19.197432 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.197351 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:19.228205 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.228150 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-676b4f5957-cwzpj" podStartSLOduration=1.492111001 podStartE2EDuration="4.228131796s" totalImagesPullingTime="2.736020795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:12:19.227602325 +0000 UTC m=+836.668066303" watchObservedRunningTime="2026-06-09 08:12:19.228131796 +0000 UTC m=+836.668595774" Jun 09 08:12:19.258628 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.258574 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f796d5f6-lfc85" podStartSLOduration=1.7095587380000001 podStartE2EDuration="4.258558664s" totalImagesPullingTime="2.548999926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:12:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:12:19.256021077 +0000 UTC m=+836.696485055" watchObservedRunningTime="2026-06-09 08:12:19.258558664 +0000 UTC m=+836.699022641" Jun 09 08:12:19.275282 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.275250 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:19.397623 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.397581 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:19.531500 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:19.531404 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:30.207728 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.207695 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:30.208178 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.207768 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:12:30.265668 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.265634 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:30.265949 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.265907 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-676b4f5957-cwzpj" podUID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" containerName="manager" containerID="cri-o://f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68" gracePeriod=30 Jun 09 08:12:30.503729 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.503702 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:30.620145 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.620104 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqn2c\" (UniqueName: \"kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c\") pod \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " Jun 09 08:12:30.620318 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.620176 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert\") pod \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\" (UID: \"9496e217-5e6b-45eb-9bf0-55c9ddcf0958\") " Jun 09 08:12:30.622418 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.622387 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c" (OuterVolumeSpecName: "kube-api-access-qqn2c") pod "9496e217-5e6b-45eb-9bf0-55c9ddcf0958" (UID: "9496e217-5e6b-45eb-9bf0-55c9ddcf0958"). InnerVolumeSpecName "kube-api-access-qqn2c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:12:30.622539 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.622391 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9496e217-5e6b-45eb-9bf0-55c9ddcf0958" (UID: "9496e217-5e6b-45eb-9bf0-55c9ddcf0958"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:12:30.721005 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.720974 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqn2c\" (UniqueName: \"kubernetes.io/projected/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-kube-api-access-qqn2c\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:12:30.721005 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:30.721006 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9496e217-5e6b-45eb-9bf0-55c9ddcf0958-webhook-cert\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:12:31.247577 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.247541 2573 generic.go:356] "Generic (PLEG): container finished" podID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" containerID="f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68" exitCode=0 Jun 09 08:12:31.247994 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.247590 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-676b4f5957-cwzpj" event={"ID":"9496e217-5e6b-45eb-9bf0-55c9ddcf0958","Type":"ContainerDied","Data":"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68"} Jun 09 08:12:31.247994 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.247633 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-676b4f5957-cwzpj" event={"ID":"9496e217-5e6b-45eb-9bf0-55c9ddcf0958","Type":"ContainerDied","Data":"bfed85677ddbdc95213e0a3b1a3459808a730f47dc5cfbcd26aad3f4717b7932"} Jun 09 08:12:31.247994 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.247654 2573 scope.go:117] "RemoveContainer" containerID="f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68" Jun 09 08:12:31.247994 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.247594 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-676b4f5957-cwzpj" Jun 09 08:12:31.255614 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.255595 2573 scope.go:117] "RemoveContainer" containerID="f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68" Jun 09 08:12:31.255935 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:12:31.255912 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68\": container with ID starting with f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68 not found: ID does not exist" containerID="f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68" Jun 09 08:12:31.256040 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.255940 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68"} err="failed to get container status \"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68\": rpc error: code = NotFound desc = could not find container \"f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68\": container with ID starting with f602b04fd9621058d2e2ed65fb77577aa62094a0128bda0049e2583644beff68 not found: ID does not exist" Jun 09 08:12:31.263398 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.263373 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:31.265724 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:31.265701 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-676b4f5957-cwzpj"] Jun 09 08:12:33.010909 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:33.010849 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" path="/var/lib/kubelet/pods/9496e217-5e6b-45eb-9bf0-55c9ddcf0958/volumes" Jun 09 08:12:52.446265 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.446231 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:52.498792 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.498762 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-854649447d-tbcjq"] Jun 09 08:12:52.499076 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.499063 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" containerName="manager" Jun 09 08:12:52.499124 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.499078 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" containerName="manager" Jun 09 08:12:52.499158 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.499128 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="9496e217-5e6b-45eb-9bf0-55c9ddcf0958" containerName="manager" Jun 09 08:12:52.502800 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.502772 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.505292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.505267 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-24qj5\"" Jun 09 08:12:52.505292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.505281 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 09 08:12:52.512097 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.512073 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-854649447d-tbcjq"] Jun 09 08:12:52.526917 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.526891 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:52.604343 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.604311 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1474308f-4d50-408a-946e-5dbafed9d45c-maas-api-tls\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.604343 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.604343 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jptjd\" (UniqueName: \"kubernetes.io/projected/1474308f-4d50-408a-946e-5dbafed9d45c-kube-api-access-jptjd\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.705832 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.705744 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1474308f-4d50-408a-946e-5dbafed9d45c-maas-api-tls\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.705832 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.705790 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jptjd\" (UniqueName: \"kubernetes.io/projected/1474308f-4d50-408a-946e-5dbafed9d45c-kube-api-access-jptjd\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.708276 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.708246 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1474308f-4d50-408a-946e-5dbafed9d45c-maas-api-tls\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.717441 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.717415 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jptjd\" (UniqueName: \"kubernetes.io/projected/1474308f-4d50-408a-946e-5dbafed9d45c-kube-api-access-jptjd\") pod \"maas-api-854649447d-tbcjq\" (UID: \"1474308f-4d50-408a-946e-5dbafed9d45c\") " pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.813795 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.813758 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:52.909318 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.909270 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:52.953712 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.953665 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-854649447d-tbcjq"] Jun 09 08:12:52.956765 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:12:52.956700 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1474308f_4d50_408a_946e_5dbafed9d45c.slice/crio-3f55fa1d8801a7e7aafcbabe6167fdbc9f75a95eb3ac6eecc0fa077739a240ee WatchSource:0}: Error finding container 3f55fa1d8801a7e7aafcbabe6167fdbc9f75a95eb3ac6eecc0fa077739a240ee: Status 404 returned error can't find the container with id 3f55fa1d8801a7e7aafcbabe6167fdbc9f75a95eb3ac6eecc0fa077739a240ee Jun 09 08:12:52.958083 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:52.958065 2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 09 08:12:53.013338 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.013306 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:12:53.203309 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.203276 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-k57g7"] Jun 09 08:12:53.205933 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.205912 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:53.208022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.207958 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 09 08:12:53.208129 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.208052 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-sb5wq\"" Jun 09 08:12:53.215100 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.215077 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-k57g7"] Jun 09 08:12:53.312021 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.311987 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2mn4\" (UniqueName: \"kubernetes.io/projected/ad33c03a-edcf-4141-96cf-f1211fa55f8d-kube-api-access-n2mn4\") pod \"payload-processing-7c6c859784-k57g7\" (UID: \"ad33c03a-edcf-4141-96cf-f1211fa55f8d\") " pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:53.334695 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.334655 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-854649447d-tbcjq" event={"ID":"1474308f-4d50-408a-946e-5dbafed9d45c","Type":"ContainerStarted","Data":"3f55fa1d8801a7e7aafcbabe6167fdbc9f75a95eb3ac6eecc0fa077739a240ee"} Jun 09 08:12:53.412552 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.412514 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2mn4\" (UniqueName: \"kubernetes.io/projected/ad33c03a-edcf-4141-96cf-f1211fa55f8d-kube-api-access-n2mn4\") pod \"payload-processing-7c6c859784-k57g7\" (UID: \"ad33c03a-edcf-4141-96cf-f1211fa55f8d\") " pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:53.421430 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.421405 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2mn4\" (UniqueName: \"kubernetes.io/projected/ad33c03a-edcf-4141-96cf-f1211fa55f8d-kube-api-access-n2mn4\") pod \"payload-processing-7c6c859784-k57g7\" (UID: \"ad33c03a-edcf-4141-96cf-f1211fa55f8d\") " pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:53.517403 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.517322 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:53.658386 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:53.658357 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-k57g7"] Jun 09 08:12:53.661342 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:12:53.661312 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad33c03a_edcf_4141_96cf_f1211fa55f8d.slice/crio-e4a16167f367aa9788bd53d013e3cb30f0605b5dc58c249ce98a92417eaf4e52 WatchSource:0}: Error finding container e4a16167f367aa9788bd53d013e3cb30f0605b5dc58c249ce98a92417eaf4e52: Status 404 returned error can't find the container with id e4a16167f367aa9788bd53d013e3cb30f0605b5dc58c249ce98a92417eaf4e52 Jun 09 08:12:54.340681 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:54.340638 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"e4a16167f367aa9788bd53d013e3cb30f0605b5dc58c249ce98a92417eaf4e52"} Jun 09 08:12:55.346348 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.346249 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"0ed35b64f3477f1481860f8a07ac13b998858d3afa4805088ba5027570efbba1"} Jun 09 08:12:55.346348 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.346326 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:12:55.347797 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.347771 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-854649447d-tbcjq" event={"ID":"1474308f-4d50-408a-946e-5dbafed9d45c","Type":"ContainerStarted","Data":"48880c19b4f5e7bbcdea69e9c6fc525579e62b3d7e8594ceb598e4a62f410083"} Jun 09 08:12:55.347951 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.347937 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:12:55.363554 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.363511 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podStartSLOduration=1.096578691 podStartE2EDuration="2.363497959s" totalImagesPullingTime="1.266919268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:12:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:12:55.362466744 +0000 UTC m=+872.802930719" watchObservedRunningTime="2026-06-09 08:12:55.363497959 +0000 UTC m=+872.803961936" Jun 09 08:12:55.377628 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:12:55.377585 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-854649447d-tbcjq" podStartSLOduration=1.406664146 podStartE2EDuration="3.377571663s" totalImagesPullingTime="1.970907517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:12:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:12:55.377003612 +0000 UTC m=+872.817467591" watchObservedRunningTime="2026-06-09 08:12:55.377571663 +0000 UTC m=+872.818035641" Jun 09 08:13:01.357158 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:01.357127 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-854649447d-tbcjq" Jun 09 08:13:03.176044 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:03.176001 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:03.176506 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:03.176243 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" containerID="cri-o://9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648" gracePeriod=30 Jun 09 08:13:04.822035 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:04.822009 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:04.911167 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:04.911065 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6h2xv\" (UniqueName: \"kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv\") pod \"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb\" (UID: \"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb\") " Jun 09 08:13:04.913221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:04.913195 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv" (OuterVolumeSpecName: "kube-api-access-6h2xv") pod "633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" (UID: "633c3b77-c78f-4eb7-8bdf-8c8783beb1cb"). InnerVolumeSpecName "kube-api-access-6h2xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:13:05.011732 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.011691 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6h2xv\" (UniqueName: \"kubernetes.io/projected/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb-kube-api-access-6h2xv\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:13:05.389042 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.389007 2573 generic.go:356] "Generic (PLEG): container finished" podID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerID="9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648" exitCode=143 Jun 09 08:13:05.389351 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.389052 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.389351 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.389096 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb","Type":"ContainerDied","Data":"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648"} Jun 09 08:13:05.389351 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.389148 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"633c3b77-c78f-4eb7-8bdf-8c8783beb1cb","Type":"ContainerDied","Data":"64f1d2c0acb91096f730e51ac7fc4baa0bf4e79a13f606525ee78ed3484ee877"} Jun 09 08:13:05.389351 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.389169 2573 scope.go:117] "RemoveContainer" containerID="9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648" Jun 09 08:13:05.398401 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.398380 2573 scope.go:117] "RemoveContainer" containerID="9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648" Jun 09 08:13:05.398653 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:13:05.398635 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648\": container with ID starting with 9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648 not found: ID does not exist" containerID="9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648" Jun 09 08:13:05.398702 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.398662 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648"} err="failed to get container status \"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648\": rpc error: code = NotFound desc = could not find container \"9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648\": container with ID starting with 9e4991fa85c9a319345759c73eaaacbd1f05bb2aa611bd559d4a02e21298b648 not found: ID does not exist" Jun 09 08:13:05.404380 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.404352 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:05.407250 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.407228 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:05.429691 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.429656 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:05.429983 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.429970 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" Jun 09 08:13:05.430047 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.429985 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" Jun 09 08:13:05.430085 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.430049 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" containerName="keycloak" Jun 09 08:13:05.432900 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.432881 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.435277 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.435257 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 09 08:13:05.435395 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.435359 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 09 08:13:05.435464 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.435419 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 09 08:13:05.435511 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.435464 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 09 08:13:05.435654 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.435530 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-g76b9\"" Jun 09 08:13:05.443747 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.443721 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:05.517667 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.517633 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skcls\" (UniqueName: \"kubernetes.io/projected/911dfe07-b0cd-4812-8bb7-9c2fce55e130-kube-api-access-skcls\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.517847 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.517702 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/911dfe07-b0cd-4812-8bb7-9c2fce55e130-test-realms\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.618206 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.618162 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/911dfe07-b0cd-4812-8bb7-9c2fce55e130-test-realms\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.618382 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.618258 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skcls\" (UniqueName: \"kubernetes.io/projected/911dfe07-b0cd-4812-8bb7-9c2fce55e130-kube-api-access-skcls\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.623529 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.619474 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/911dfe07-b0cd-4812-8bb7-9c2fce55e130-test-realms\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.628770 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.628739 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skcls\" (UniqueName: \"kubernetes.io/projected/911dfe07-b0cd-4812-8bb7-9c2fce55e130-kube-api-access-skcls\") pod \"maas-keycloak-0\" (UID: \"911dfe07-b0cd-4812-8bb7-9c2fce55e130\") " pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.742731 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.742687 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:05.880238 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:05.880204 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 09 08:13:05.882881 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:13:05.882838 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911dfe07_b0cd_4812_8bb7_9c2fce55e130.slice/crio-9396d8add74c74297db49c25ba50246bbbc7f196a3e75871529b2a7edecee080 WatchSource:0}: Error finding container 9396d8add74c74297db49c25ba50246bbbc7f196a3e75871529b2a7edecee080: Status 404 returned error can't find the container with id 9396d8add74c74297db49c25ba50246bbbc7f196a3e75871529b2a7edecee080 Jun 09 08:13:06.354285 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.354255 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:13:06.396830 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.396788 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"911dfe07-b0cd-4812-8bb7-9c2fce55e130","Type":"ContainerStarted","Data":"eb874b699d0747dcb19c558e535468e3606638a31da03f7c60173719500b0d0f"} Jun 09 08:13:06.397012 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.396851 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"911dfe07-b0cd-4812-8bb7-9c2fce55e130","Type":"ContainerStarted","Data":"9396d8add74c74297db49c25ba50246bbbc7f196a3e75871529b2a7edecee080"} Jun 09 08:13:06.417925 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.417835 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.072044158 podStartE2EDuration="1.417809111s" totalImagesPullingTime="345.764953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:13:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:13:06.412681966 +0000 UTC m=+883.853145945" watchObservedRunningTime="2026-06-09 08:13:06.417809111 +0000 UTC m=+883.858273090" Jun 09 08:13:06.743791 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.743749 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:06.745355 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:06.745316 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:07.013509 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:07.013417 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="633c3b77-c78f-4eb7-8bdf-8c8783beb1cb" path="/var/lib/kubelet/pods/633c3b77-c78f-4eb7-8bdf-8c8783beb1cb/volumes" Jun 09 08:13:07.743653 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:07.743593 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:08.743431 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:08.743385 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:09.744197 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:09.744077 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:10.743804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:10.743760 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:11.743530 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:11.743465 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:12.744041 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:12.743992 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:13.743927 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:13.743835 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:14.743757 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:14.743706 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:15.743663 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:15.743571 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:15.744022 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:15.743988 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:16.743856 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:16.743798 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:17.743464 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:17.743390 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:18.743225 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:18.743163 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:19.744025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:19.743973 2573 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="Get \"http://10.133.0.27:9000/health/started\": dial tcp 10.133.0.27:9000: connect: connection refused" Jun 09 08:13:20.843584 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:20.842568 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:20.859174 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:20.859118 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 08:13:30.847965 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:30.847900 2573 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="911dfe07-b0cd-4812-8bb7-9c2fce55e130" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 09 08:13:40.849565 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:40.849517 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 09 08:13:47.270102 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:47.270061 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:13:53.202503 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:53.202461 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:13:53.304068 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:53.304024 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:13:54.003271 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:54.003235 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:13:56.297887 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.297835 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm"] Jun 09 08:13:56.301273 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.301250 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.303265 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.303236 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 09 08:13:56.303409 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.303237 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-qgdvr\"" Jun 09 08:13:56.303786 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.303769 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 09 08:13:56.303965 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.303935 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 09 08:13:56.315034 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.315006 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm"] Jun 09 08:13:56.335134 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.335095 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:13:56.373258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373222 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.373258 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373258 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjjkq\" (UniqueName: \"kubernetes.io/projected/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kube-api-access-cjjkq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.373484 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373284 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.373484 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373431 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.373484 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373459 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8687006a-ef0a-4482-9b6c-b5c71b6ad021-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.373587 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.373520 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474386 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474350 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474591 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474393 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjjkq\" (UniqueName: \"kubernetes.io/projected/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kube-api-access-cjjkq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474591 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474415 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474591 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474480 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474591 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8687006a-ef0a-4482-9b6c-b5c71b6ad021-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474591 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474550 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474892 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474833 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474963 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474914 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.474963 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.474944 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.476801 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.476779 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8687006a-ef0a-4482-9b6c-b5c71b6ad021-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.477123 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.477103 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8687006a-ef0a-4482-9b6c-b5c71b6ad021-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.483398 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.483370 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjjkq\" (UniqueName: \"kubernetes.io/projected/8687006a-ef0a-4482-9b6c-b5c71b6ad021-kube-api-access-cjjkq\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm\" (UID: \"8687006a-ef0a-4482-9b6c-b5c71b6ad021\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.611453 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.611368 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:13:56.746025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:56.745991 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm"] Jun 09 08:13:56.749272 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:13:56.749243 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8687006a_ef0a_4482_9b6c_b5c71b6ad021.slice/crio-015261e89fb18e69c32c67dc7af511822695b834934863d54f89111efb044ef4 WatchSource:0}: Error finding container 015261e89fb18e69c32c67dc7af511822695b834934863d54f89111efb044ef4: Status 404 returned error can't find the container with id 015261e89fb18e69c32c67dc7af511822695b834934863d54f89111efb044ef4 Jun 09 08:13:57.645778 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:57.645731 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" event={"ID":"8687006a-ef0a-4482-9b6c-b5c71b6ad021","Type":"ContainerStarted","Data":"015261e89fb18e69c32c67dc7af511822695b834934863d54f89111efb044ef4"} Jun 09 08:13:59.322343 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:13:59.322300 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:01.666041 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:01.666002 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" event={"ID":"8687006a-ef0a-4482-9b6c-b5c71b6ad021","Type":"ContainerStarted","Data":"5a4b2fc3180421f670a33c650aca2ec1a0b8b9a25fa0c86d99e3cc5229d42940"} Jun 09 08:14:02.672025 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:02.671987 2573 generic.go:356] "Generic (PLEG): container finished" podID="8687006a-ef0a-4482-9b6c-b5c71b6ad021" containerID="5a4b2fc3180421f670a33c650aca2ec1a0b8b9a25fa0c86d99e3cc5229d42940" exitCode=0 Jun 09 08:14:02.672405 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:02.672035 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" event={"ID":"8687006a-ef0a-4482-9b6c-b5c71b6ad021","Type":"ContainerDied","Data":"5a4b2fc3180421f670a33c650aca2ec1a0b8b9a25fa0c86d99e3cc5229d42940"} Jun 09 08:14:04.685880 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:04.685831 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" event={"ID":"8687006a-ef0a-4482-9b6c-b5c71b6ad021","Type":"ContainerStarted","Data":"cd5091b18f1d3ed715fdcb4a7d0b566972b2f312047367b0164ee5c62dd18343"} Jun 09 08:14:04.707110 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:04.707051 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" podStartSLOduration=2.38073896 podStartE2EDuration="8.707033779s" totalImagesPullingTime="4.985511187s" totalInitContainerRuntime="1.340783632s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:13:56 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:14:04.706319999 +0000 UTC m=+942.146783978" watchObservedRunningTime="2026-06-09 08:14:04.707033779 +0000 UTC m=+942.147497758" Jun 09 08:14:06.612544 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:06.612504 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:14:06.612984 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:06.612573 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:14:06.625397 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:06.625364 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:14:06.705085 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:06.705051 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm" Jun 09 08:14:08.779226 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.779191 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx"] Jun 09 08:14:08.786157 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.786131 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.789103 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.789074 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 09 08:14:08.797248 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.797209 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx"] Jun 09 08:14:08.814035 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.814004 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:08.883715 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.883664 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ce3a7c-f083-4231-bc13-6fcc657551c3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.883715 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.883709 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.883985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.883805 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.883985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.883852 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.883985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.883965 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfkl5\" (UniqueName: \"kubernetes.io/projected/33ce3a7c-f083-4231-bc13-6fcc657551c3-kube-api-access-hfkl5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.884108 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.884062 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985466 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985417 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ce3a7c-f083-4231-bc13-6fcc657551c3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985466 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985499 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985545 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfkl5\" (UniqueName: \"kubernetes.io/projected/33ce3a7c-f083-4231-bc13-6fcc657551c3-kube-api-access-hfkl5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.985680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.985579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.986029 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.986003 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.986121 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.986043 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.986121 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.986078 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.987956 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.987936 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/33ce3a7c-f083-4231-bc13-6fcc657551c3-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.988228 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.988205 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/33ce3a7c-f083-4231-bc13-6fcc657551c3-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:08.994327 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:08.994301 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfkl5\" (UniqueName: \"kubernetes.io/projected/33ce3a7c-f083-4231-bc13-6fcc657551c3-kube-api-access-hfkl5\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx\" (UID: \"33ce3a7c-f083-4231-bc13-6fcc657551c3\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:09.098254 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:09.098146 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:09.232344 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:09.232269 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx"] Jun 09 08:14:09.234613 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:14:09.234576 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33ce3a7c_f083_4231_bc13_6fcc657551c3.slice/crio-d4574190b042438d42567aff617b22cf2cab39f56a0fa11c04fd7e8396f4247e WatchSource:0}: Error finding container d4574190b042438d42567aff617b22cf2cab39f56a0fa11c04fd7e8396f4247e: Status 404 returned error can't find the container with id d4574190b042438d42567aff617b22cf2cab39f56a0fa11c04fd7e8396f4247e Jun 09 08:14:09.707883 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:09.707830 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" event={"ID":"33ce3a7c-f083-4231-bc13-6fcc657551c3","Type":"ContainerStarted","Data":"a384ba56ef027afb00b8aad0a41c4cd47883f32ae7e49afb8f6c6451c71f31c6"} Jun 09 08:14:09.708097 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:09.707945 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" event={"ID":"33ce3a7c-f083-4231-bc13-6fcc657551c3","Type":"ContainerStarted","Data":"d4574190b042438d42567aff617b22cf2cab39f56a0fa11c04fd7e8396f4247e"} Jun 09 08:14:10.713701 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:10.713665 2573 generic.go:356] "Generic (PLEG): container finished" podID="33ce3a7c-f083-4231-bc13-6fcc657551c3" containerID="a384ba56ef027afb00b8aad0a41c4cd47883f32ae7e49afb8f6c6451c71f31c6" exitCode=0 Jun 09 08:14:10.714126 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:10.713746 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" event={"ID":"33ce3a7c-f083-4231-bc13-6fcc657551c3","Type":"ContainerDied","Data":"a384ba56ef027afb00b8aad0a41c4cd47883f32ae7e49afb8f6c6451c71f31c6"} Jun 09 08:14:11.412339 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:11.412243 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:11.720559 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:11.720520 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" event={"ID":"33ce3a7c-f083-4231-bc13-6fcc657551c3","Type":"ContainerStarted","Data":"b60096ea703a21d3f775a200f56deaed281ac342bdc59eb0307c285dea50f9d2"} Jun 09 08:14:11.737332 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:11.737279 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" podStartSLOduration=2.244629553 podStartE2EDuration="3.737263557s" totalImagesPullingTime="225.423208ms" totalInitContainerRuntime="1.267210796s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:14:11.736591673 +0000 UTC m=+949.177055654" watchObservedRunningTime="2026-06-09 08:14:11.737263557 +0000 UTC m=+949.177727535" Jun 09 08:14:17.484461 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:17.484426 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:17.585288 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:17.585252 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:19.098438 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.098404 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:19.098910 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.098697 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:19.111393 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.111368 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:19.763152 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.763119 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx" Jun 09 08:14:19.965679 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.965637 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv"] Jun 09 08:14:19.970199 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.970156 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.972382 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.972358 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 09 08:14:19.973727 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973706 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.973857 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973738 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad07049-7762-440a-a536-0069ec3300d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.973857 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973787 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.973857 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973823 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r55s\" (UniqueName: \"kubernetes.io/projected/bad07049-7762-440a-a536-0069ec3300d0-kube-api-access-4r55s\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.974049 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973929 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.974049 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.973981 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:19.984434 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:19.984410 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv"] Jun 09 08:14:20.012427 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.012394 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:20.074635 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074542 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.074635 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074579 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r55s\" (UniqueName: \"kubernetes.io/projected/bad07049-7762-440a-a536-0069ec3300d0-kube-api-access-4r55s\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.074635 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074616 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.074934 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074642 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.074934 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074674 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.074934 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.074801 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad07049-7762-440a-a536-0069ec3300d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.075091 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.075020 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.075091 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.075048 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.075091 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.075068 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.076930 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.076906 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bad07049-7762-440a-a536-0069ec3300d0-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.077242 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.077225 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bad07049-7762-440a-a536-0069ec3300d0-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.082585 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.082564 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r55s\" (UniqueName: \"kubernetes.io/projected/bad07049-7762-440a-a536-0069ec3300d0-kube-api-access-4r55s\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv\" (UID: \"bad07049-7762-440a-a536-0069ec3300d0\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.281471 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.281437 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:20.425223 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.425194 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv"] Jun 09 08:14:20.427050 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:14:20.427011 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbad07049_7762_440a_a536_0069ec3300d0.slice/crio-663c450254191cc7b2f047f5803f7ba222787d56dc66437c8e27ec614f1cf1bc WatchSource:0}: Error finding container 663c450254191cc7b2f047f5803f7ba222787d56dc66437c8e27ec614f1cf1bc: Status 404 returned error can't find the container with id 663c450254191cc7b2f047f5803f7ba222787d56dc66437c8e27ec614f1cf1bc Jun 09 08:14:20.758214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.758177 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" event={"ID":"bad07049-7762-440a-a536-0069ec3300d0","Type":"ContainerStarted","Data":"ddbf8bd9f5f76dbc967ab434d7e01d2f8dc5a2403b39cc37c86fa6307e6a2bad"} Jun 09 08:14:20.758409 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:20.758241 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" event={"ID":"bad07049-7762-440a-a536-0069ec3300d0","Type":"ContainerStarted","Data":"663c450254191cc7b2f047f5803f7ba222787d56dc66437c8e27ec614f1cf1bc"} Jun 09 08:14:24.776895 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:24.776841 2573 generic.go:356] "Generic (PLEG): container finished" podID="bad07049-7762-440a-a536-0069ec3300d0" containerID="ddbf8bd9f5f76dbc967ab434d7e01d2f8dc5a2403b39cc37c86fa6307e6a2bad" exitCode=0 Jun 09 08:14:24.777271 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:24.776916 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" event={"ID":"bad07049-7762-440a-a536-0069ec3300d0","Type":"ContainerDied","Data":"ddbf8bd9f5f76dbc967ab434d7e01d2f8dc5a2403b39cc37c86fa6307e6a2bad"} Jun 09 08:14:25.783320 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:25.783281 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" event={"ID":"bad07049-7762-440a-a536-0069ec3300d0","Type":"ContainerStarted","Data":"242788fa33f8373ab9dc00ee807dbd9f585f4ad465afce0d6ad6a8fb2498c022"} Jun 09 08:14:25.803260 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:25.803207 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" podStartSLOduration=3.123983338 podStartE2EDuration="6.803189999s" totalImagesPullingTime="209.579919ms" totalInitContainerRuntime="3.469626742s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:14:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:14:25.80138184 +0000 UTC m=+963.241845818" watchObservedRunningTime="2026-06-09 08:14:25.803189999 +0000 UTC m=+963.243653976" Jun 09 08:14:28.094344 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.094312 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq"] Jun 09 08:14:28.098189 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.098167 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.100255 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.100233 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 09 08:14:28.108371 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.108348 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq"] Jun 09 08:14:28.123873 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.123836 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:14:28.139962 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.139934 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.140044 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.139970 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.140044 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.139989 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.140118 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.140100 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4574e725-d6a4-40ba-84a5-0b9b77af1d98-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.140156 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.140139 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.140190 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.140172 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptnfw\" (UniqueName: \"kubernetes.io/projected/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kube-api-access-ptnfw\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241506 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241466 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4574e725-d6a4-40ba-84a5-0b9b77af1d98-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241520 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241561 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptnfw\" (UniqueName: \"kubernetes.io/projected/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kube-api-access-ptnfw\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241603 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241629 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.241904 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.241753 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.242099 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.242067 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.242214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.242122 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-home\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.242214 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.242149 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-model-cache\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.243902 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.243856 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4574e725-d6a4-40ba-84a5-0b9b77af1d98-dshm\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.244637 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.244616 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4574e725-d6a4-40ba-84a5-0b9b77af1d98-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.249550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.249529 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptnfw\" (UniqueName: \"kubernetes.io/projected/4574e725-d6a4-40ba-84a5-0b9b77af1d98-kube-api-access-ptnfw\") pod \"premium-simulated-simulated-premium-kserve-66967c79b-hzgnq\" (UID: \"4574e725-d6a4-40ba-84a5-0b9b77af1d98\") " pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.407683 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.407592 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:28.534428 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.534401 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq"] Jun 09 08:14:28.536662 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:14:28.536632 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4574e725_d6a4_40ba_84a5_0b9b77af1d98.slice/crio-4bc6f8915bd35c544b2ee9988df09a6e6e015ebcf543b7e6394bda7340aafa02 WatchSource:0}: Error finding container 4bc6f8915bd35c544b2ee9988df09a6e6e015ebcf543b7e6394bda7340aafa02: Status 404 returned error can't find the container with id 4bc6f8915bd35c544b2ee9988df09a6e6e015ebcf543b7e6394bda7340aafa02 Jun 09 08:14:28.799480 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.799439 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" event={"ID":"4574e725-d6a4-40ba-84a5-0b9b77af1d98","Type":"ContainerStarted","Data":"db777cf87b60741bd5e884a77e40007edf584989b5ec5b6b971e614dd0a55c0f"} Jun 09 08:14:28.799652 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:28.799490 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" event={"ID":"4574e725-d6a4-40ba-84a5-0b9b77af1d98","Type":"ContainerStarted","Data":"4bc6f8915bd35c544b2ee9988df09a6e6e015ebcf543b7e6394bda7340aafa02"} Jun 09 08:14:30.282321 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.282286 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:30.282698 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.282352 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:30.296163 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.296135 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:30.809626 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.809591 2573 generic.go:356] "Generic (PLEG): container finished" podID="4574e725-d6a4-40ba-84a5-0b9b77af1d98" containerID="db777cf87b60741bd5e884a77e40007edf584989b5ec5b6b971e614dd0a55c0f" exitCode=0 Jun 09 08:14:30.809796 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.809666 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" event={"ID":"4574e725-d6a4-40ba-84a5-0b9b77af1d98","Type":"ContainerDied","Data":"db777cf87b60741bd5e884a77e40007edf584989b5ec5b6b971e614dd0a55c0f"} Jun 09 08:14:30.821102 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:30.821077 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv" Jun 09 08:14:31.821001 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:31.820961 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" event={"ID":"4574e725-d6a4-40ba-84a5-0b9b77af1d98","Type":"ContainerStarted","Data":"511982d8d59990b373cc3050d01c1aec91103af05b3ea6aea5a841f9224aec86"} Jun 09 08:14:31.840698 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:31.840647 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" podStartSLOduration=2.164718566 podStartE2EDuration="3.840632155s" totalImagesPullingTime="383.130142ms" totalInitContainerRuntime="1.292783447s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:14:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:14:31.839609971 +0000 UTC m=+969.280073949" watchObservedRunningTime="2026-06-09 08:14:31.840632155 +0000 UTC m=+969.281096132" Jun 09 08:14:38.407966 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:38.407918 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:38.408347 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:38.407985 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:38.420526 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:38.420500 2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:38.862209 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:38.862179 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-66967c79b-hzgnq" Jun 09 08:14:55.927580 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:55.927545 2573 generic.go:356] "Generic (PLEG): container finished" podID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" containerID="0ed35b64f3477f1481860f8a07ac13b998858d3afa4805088ba5027570efbba1" exitCode=1 Jun 09 08:14:55.928064 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:55.927620 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerDied","Data":"0ed35b64f3477f1481860f8a07ac13b998858d3afa4805088ba5027570efbba1"} Jun 09 08:14:55.928064 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:55.927980 2573 scope.go:117] "RemoveContainer" containerID="0ed35b64f3477f1481860f8a07ac13b998858d3afa4805088ba5027570efbba1" Jun 09 08:14:56.352648 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:56.352546 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:14:56.933356 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:56.933315 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e"} Jun 09 08:14:56.933738 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:14:56.933504 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:15:07.940081 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:15:07.940051 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:15:25.826710 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:15:25.826659 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:15:25.916181 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:15:25.916145 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:15:26.900357 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:15:26.900325 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:15:27.000880 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:15:27.000826 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:16:32.267057 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:32.266969 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:16:52.469007 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.468942 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:16:52.557013 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.556977 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:16:52.557257 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.557231 2573 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7f796d5f6-lfc85" podUID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" containerName="manager" containerID="cri-o://34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77" gracePeriod=30 Jun 09 08:16:52.797249 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.797219 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:16:52.800221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.800194 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert\") pod \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " Jun 09 08:16:52.800345 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.800290 2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgr4p\" (UniqueName: \"kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p\") pod \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\" (UID: \"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3\") " Jun 09 08:16:52.802163 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.802134 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" (UID: "8ae0ebf7-19ce-48e2-88e2-c10127bd28f3"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 09 08:16:52.802263 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.802214 2573 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p" (OuterVolumeSpecName: "kube-api-access-xgr4p") pod "8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" (UID: "8ae0ebf7-19ce-48e2-88e2-c10127bd28f3"). InnerVolumeSpecName "kube-api-access-xgr4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 09 08:16:52.901650 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.901621 2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-webhook-cert\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:16:52.901650 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:52.901648 2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgr4p\" (UniqueName: \"kubernetes.io/projected/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3-kube-api-access-xgr4p\") on node \"ip-10-0-141-98.ec2.internal\" DevicePath \"\"" Jun 09 08:16:53.453771 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.453738 2573 generic.go:356] "Generic (PLEG): container finished" podID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" containerID="34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77" exitCode=0 Jun 09 08:16:53.453973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.453779 2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-lfc85" Jun 09 08:16:53.453973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.453809 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-lfc85" event={"ID":"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3","Type":"ContainerDied","Data":"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77"} Jun 09 08:16:53.453973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.453849 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-lfc85" event={"ID":"8ae0ebf7-19ce-48e2-88e2-c10127bd28f3","Type":"ContainerDied","Data":"4f0943f0b8f56628b83b27cecba18fb319897eada08243d2361d507711a2eaca"} Jun 09 08:16:53.453973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.453899 2573 scope.go:117] "RemoveContainer" containerID="34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77" Jun 09 08:16:53.462281 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.462258 2573 scope.go:117] "RemoveContainer" containerID="34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77" Jun 09 08:16:53.462562 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:16:53.462540 2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77\": container with ID starting with 34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77 not found: ID does not exist" containerID="34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77" Jun 09 08:16:53.462619 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.462569 2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77"} err="failed to get container status \"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77\": rpc error: code = NotFound desc = could not find container \"34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77\": container with ID starting with 34157e731a17a8d94dc4f93f4cb6fdc77cd26e1ac1feec42407f9e669a916d77 not found: ID does not exist" Jun 09 08:16:53.470897 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.470855 2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:16:53.479804 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:53.479782 2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-lfc85"] Jun 09 08:16:54.118747 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.118710 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:16:54.219700 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.219665 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7f796d5f6-hhxc6"] Jun 09 08:16:54.220041 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.220027 2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" containerName="manager" Jun 09 08:16:54.220092 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.220042 2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" containerName="manager" Jun 09 08:16:54.220128 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.220110 2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" containerName="manager" Jun 09 08:16:54.224206 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.224190 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.226150 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.226127 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 09 08:16:54.226908 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.226890 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-zxkcm\"" Jun 09 08:16:54.226975 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.226899 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 09 08:16:54.230234 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.230214 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-hhxc6"] Jun 09 08:16:54.313412 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.313369 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58ee65a8-0ae3-4dae-8891-dda166151679-webhook-cert\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.313590 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.313429 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnz5f\" (UniqueName: \"kubernetes.io/projected/58ee65a8-0ae3-4dae-8891-dda166151679-kube-api-access-wnz5f\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.414055 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.413984 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58ee65a8-0ae3-4dae-8891-dda166151679-webhook-cert\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.414055 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.414034 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wnz5f\" (UniqueName: \"kubernetes.io/projected/58ee65a8-0ae3-4dae-8891-dda166151679-kube-api-access-wnz5f\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.416497 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.416467 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/58ee65a8-0ae3-4dae-8891-dda166151679-webhook-cert\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.422316 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.422291 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnz5f\" (UniqueName: \"kubernetes.io/projected/58ee65a8-0ae3-4dae-8891-dda166151679-kube-api-access-wnz5f\") pod \"maas-controller-7f796d5f6-hhxc6\" (UID: \"58ee65a8-0ae3-4dae-8891-dda166151679\") " pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.536264 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.536230 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:54.661610 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:54.661544 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7f796d5f6-hhxc6"] Jun 09 08:16:54.664132 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:16:54.664066 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ee65a8_0ae3_4dae_8891_dda166151679.slice/crio-4258c01c291849c19530a53db27e1365a97c1434c62bbe06b309ec58b4d1c659 WatchSource:0}: Error finding container 4258c01c291849c19530a53db27e1365a97c1434c62bbe06b309ec58b4d1c659: Status 404 returned error can't find the container with id 4258c01c291849c19530a53db27e1365a97c1434c62bbe06b309ec58b4d1c659 Jun 09 08:16:55.011744 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:55.011707 2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ae0ebf7-19ce-48e2-88e2-c10127bd28f3" path="/var/lib/kubelet/pods/8ae0ebf7-19ce-48e2-88e2-c10127bd28f3/volumes" Jun 09 08:16:55.466619 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:55.466583 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" event={"ID":"58ee65a8-0ae3-4dae-8891-dda166151679","Type":"ContainerStarted","Data":"7ec1e02d58a4b950515f6d4bd898d2ee2db29d55d7f642f815703cb82b2bac6d"} Jun 09 08:16:55.466788 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:55.466664 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" event={"ID":"58ee65a8-0ae3-4dae-8891-dda166151679","Type":"ContainerStarted","Data":"4258c01c291849c19530a53db27e1365a97c1434c62bbe06b309ec58b4d1c659"} Jun 09 08:16:55.466788 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:55.466741 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:16:55.483212 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:55.483163 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" podStartSLOduration=1.108746738 podStartE2EDuration="1.483150609s" totalImagesPullingTime="374.403871ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-09 08:16:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:16:55.48180447 +0000 UTC m=+1112.922268448" watchObservedRunningTime="2026-06-09 08:16:55.483150609 +0000 UTC m=+1112.923614586" Jun 09 08:16:56.473151 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:56.473120 2573 generic.go:356] "Generic (PLEG): container finished" podID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" containerID="3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e" exitCode=1 Jun 09 08:16:56.473579 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:56.473191 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerDied","Data":"3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e"} Jun 09 08:16:56.473579 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:56.473252 2573 scope.go:117] "RemoveContainer" containerID="0ed35b64f3477f1481860f8a07ac13b998858d3afa4805088ba5027570efbba1" Jun 09 08:16:56.473679 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:56.473631 2573 scope.go:117] "RemoveContainer" containerID="3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e" Jun 09 08:16:56.473877 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:16:56.473841 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:16:57.938799 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:57.938763 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:16:57.939246 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:16:57.939121 2573 scope.go:117] "RemoveContainer" containerID="3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e" Jun 09 08:16:57.939316 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:16:57.939297 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:17:06.479886 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:06.479840 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7f796d5f6-hhxc6" Jun 09 08:17:12.005578 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:12.005545 2573 scope.go:117] "RemoveContainer" containerID="3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e" Jun 09 08:17:12.546499 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:12.546461 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4"} Jun 09 08:17:12.546699 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:12.546680 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:17:16.472169 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:16.472131 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:16.573070 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:16.573035 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:23.552965 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:23.552929 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:17:35.981416 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:35.981358 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:36.108255 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:36.108220 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:44.421244 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:44.421201 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:50.696803 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:50.696766 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:17:50.796128 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:17:50.796090 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:22.812266 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:22.812223 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:22.911430 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:22.911397 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:41.358722 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:41.358685 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:46.772219 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:46.772177 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:52.887737 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:52.887697 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:18:54.088075 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:18:54.088032 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:09.781900 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:09.781796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:09.892178 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:09.892139 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:09.992756 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:09.992724 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:13.072096 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.072061 2573 generic.go:356] "Generic (PLEG): container finished" podID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" exitCode=1 Jun 09 08:19:13.072642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.072140 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerDied","Data":"83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4"} Jun 09 08:19:13.072642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.072218 2573 scope.go:117] "RemoveContainer" containerID="3dd0af05869db3b8d3edaad11a1cf5f1492cb88f86b1f13b5f19bb228f59870e" Jun 09 08:19:13.072642 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.072512 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:19:13.072816 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:19:13.072760 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:19:13.518009 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.517972 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:19:13.551651 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:13.551620 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:19:14.078469 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:14.078435 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:19:14.078843 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:19:14.078643 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:19:15.082350 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:15.082323 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:19:15.082713 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:19:15.082549 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:19:16.045829 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:16.045798 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:16.144491 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:16.144446 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:24.186149 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:24.186101 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:24.286705 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:24.286675 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:24.386125 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:24.386089 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:24.486422 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:24.486332 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:26.005556 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:26.005526 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:19:26.005936 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:19:26.005723 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:19:39.006272 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:39.006237 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:19:39.188122 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:39.188083 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294"} Jun 09 08:19:39.188331 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:39.188311 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:19:40.899708 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:40.899670 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:41.017292 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:41.017257 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:41.123391 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:41.123355 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:47.277417 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:47.277378 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:47.333646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:47.333602 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:19:50.194307 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:19:50.194274 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:20:07.574739 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:07.574696 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:07.683117 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:07.683078 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:07.777705 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:07.777673 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:13.693650 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:13.693599 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:13.768610 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:13.768573 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:13.865197 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:13.865166 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:13.960783 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:13.960695 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:14.061999 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:14.061965 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:19.949336 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:19.949292 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:20:20.010837 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:20:20.010796 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:21:39.718943 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:39.718907 2573 generic.go:356] "Generic (PLEG): container finished" podID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" exitCode=1 Jun 09 08:21:39.719410 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:39.718949 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerDied","Data":"4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294"} Jun 09 08:21:39.719410 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:39.719023 2573 scope.go:117] "RemoveContainer" containerID="83bb7f8d8572473ee0120b62e072ade3a617faee85e5274abcdf6af361b8bfa4" Jun 09 08:21:39.719410 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:39.719303 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:21:39.719594 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:21:39.719498 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:21:40.192858 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:40.192822 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:21:40.724618 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:40.724595 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:21:40.725044 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:21:40.724849 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:21:47.629981 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:47.629938 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:21:47.696973 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:47.696937 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:21:50.959550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:50.959510 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:21:53.517944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:53.517916 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:21:53.518349 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:21:53.518241 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:21:53.518419 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:21:53.518399 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:22:05.006232 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:22:05.006199 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:22:05.006615 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:22:05.006436 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:22:20.005933 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:22:20.005845 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:22:20.897775 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:22:20.897738 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014"} Jun 09 08:22:20.897961 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:22:20.897939 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:22:31.903731 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:22:31.903698 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:24:20.422157 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:20.422125 2573 generic.go:356] "Generic (PLEG): container finished" podID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" exitCode=1 Jun 09 08:24:20.422617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:20.422194 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerDied","Data":"f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014"} Jun 09 08:24:20.422617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:20.422253 2573 scope.go:117] "RemoveContainer" containerID="4f2873cad8675473ee512e19b8df14637c83a1ea2bcd89ec698745d037b4d294" Jun 09 08:24:20.422617 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:20.422550 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:20.422758 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:20.422739 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:24:21.902785 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:21.902740 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:24:21.903231 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:21.903139 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:21.903390 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:21.903369 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:24:33.007907 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:33.007842 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:33.008275 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:33.008177 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:24:33.517719 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:33.517683 2573 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:24:33.518080 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:33.518065 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:33.518289 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:33.518267 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:24:45.006536 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:45.006509 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:45.006949 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:45.006719 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:24:57.005684 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:24:57.005654 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:24:57.006181 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:24:57.005891 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:25:10.006004 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:10.005921 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:25:10.006404 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:25:10.006167 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:25:14.432242 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:14.432187 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-854649447d-tbcjq_1474308f-4d50-408a-946e-5dbafed9d45c/maas-api/0.log" Jun 09 08:25:14.698990 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:14.698887 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f796d5f6-hhxc6_58ee65a8-0ae3-4dae-8891-dda166151679/manager/0.log" Jun 09 08:25:15.269315 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:15.269284 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6b94fd7b-s497m_f4d60053-760c-4d10-b0aa-f08e930c8820/copy-manifests/0.log" Jun 09 08:25:15.277261 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:15.277231 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6b94fd7b-s497m_f4d60053-760c-4d10-b0aa-f08e930c8820/manager/0.log" Jun 09 08:25:15.584570 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:15.584477 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-9cprz_5f5b57c4-23fe-4ef1-b3ae-145efe8025cc/postgres/0.log" Jun 09 08:25:17.402682 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:17.402650 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dd968_c2dc76d0-4c62-4cf1-b319-13af33c9fa4b/kuadrant-console-plugin/0.log" Jun 09 08:25:18.551481 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:18.551448 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69895d85c6-x79h4_3bafe8a0-1173-4ad3-afc2-054f00a90e7b/kube-auth-proxy/0.log" Jun 09 08:25:18.922680 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:18.922646 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:19.379397 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.379312 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm_8687006a-ef0a-4482-9b6c-b5c71b6ad021/storage-initializer/0.log" Jun 09 08:25:19.386679 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.386654 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-gptgm_8687006a-ef0a-4482-9b6c-b5c71b6ad021/main/0.log" Jun 09 08:25:19.613097 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.613066 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx_33ce3a7c-f083-4231-bc13-6fcc657551c3/storage-initializer/0.log" Jun 09 08:25:19.620453 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.620425 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-q9dvx_33ce3a7c-f083-4231-bc13-6fcc657551c3/main/0.log" Jun 09 08:25:19.851134 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.851093 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv_bad07049-7762-440a-a536-0069ec3300d0/storage-initializer/0.log" Jun 09 08:25:19.858310 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.858286 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-zx6sv_bad07049-7762-440a-a536-0069ec3300d0/main/0.log" Jun 09 08:25:19.971078 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.971038 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-hzgnq_4574e725-d6a4-40ba-84a5-0b9b77af1d98/storage-initializer/0.log" Jun 09 08:25:19.978070 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:19.978041 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-66967c79b-hzgnq_4574e725-d6a4-40ba-84a5-0b9b77af1d98/main/0.log" Jun 09 08:25:22.006240 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:22.006202 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:25:22.006627 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:25:22.006437 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:25:24.597205 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:24.597171 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:25:25.791619 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:25.791577 2573 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" limit=1024 Jun 09 08:25:27.555273 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:27.555241 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-glwhj_9d7869c1-1935-42f4-8ff3-3b4ecdb49ff4/global-pull-secret-syncer/0.log" Jun 09 08:25:27.640342 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:27.640304 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5tn2p_dce1b52b-882e-4217-8fda-5317c83d4712/konnectivity-agent/0.log" Jun 09 08:25:27.756428 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:27.756400 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-98.ec2.internal_9efa5c0d29862f4f6670610d25ee3730/haproxy/0.log" Jun 09 08:25:32.978885 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:32.978829 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-dd968_c2dc76d0-4c62-4cf1-b319-13af33c9fa4b/kuadrant-console-plugin/0.log" Jun 09 08:25:34.006254 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:34.006217 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:25:34.006650 ip-10-0-141-98 kubenswrapper[2573]: E0609 08:25:34.006437 2573 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-k57g7_openshift-ingress(ad33c03a-edcf-4141-96cf-f1211fa55f8d)\"" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" podUID="ad33c03a-edcf-4141-96cf-f1211fa55f8d" Jun 09 08:25:35.088999 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.088918 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mstf4_b121b8f8-011d-4715-b3e2-e6e2563f28a1/node-exporter/0.log" Jun 09 08:25:35.107662 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.107631 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mstf4_b121b8f8-011d-4715-b3e2-e6e2563f28a1/kube-rbac-proxy/0.log" Jun 09 08:25:35.128721 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.128692 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-mstf4_b121b8f8-011d-4715-b3e2-e6e2563f28a1/init-textfile/0.log" Jun 09 08:25:35.326327 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.326295 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/prometheus/0.log" Jun 09 08:25:35.345698 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.345603 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/config-reloader/0.log" Jun 09 08:25:35.365048 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.365010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/thanos-sidecar/0.log" Jun 09 08:25:35.386886 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.386845 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/kube-rbac-proxy-web/0.log" Jun 09 08:25:35.406459 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.406434 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/kube-rbac-proxy/0.log" Jun 09 08:25:35.423798 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.423764 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/kube-rbac-proxy-thanos/0.log" Jun 09 08:25:35.455931 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:35.455896 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_78344f2f-58a8-4f5a-bc56-4da6299edc55/init-config-reloader/0.log" Jun 09 08:25:38.914404 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.914366 2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj"] Jun 09 08:25:38.916852 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.916827 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:38.919002 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.918971 2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qzrcm\"/\"default-dockercfg-7fn9v\"" Jun 09 08:25:38.919255 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.919232 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzrcm\"/\"openshift-service-ca.crt\"" Jun 09 08:25:38.919705 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.919687 2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qzrcm\"/\"kube-root-ca.crt\"" Jun 09 08:25:38.929134 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:38.929110 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj"] Jun 09 08:25:39.005498 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.005448 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5b6d\" (UniqueName: \"kubernetes.io/projected/ecbba608-dfd3-4175-ae83-bf62c402bb80-kube-api-access-k5b6d\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.005674 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.005507 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-sys\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.005764 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.005740 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-proc\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.005805 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.005786 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-podres\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.005839 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.005805 2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-lib-modules\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.106985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.106942 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-podres\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.106985 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.106985 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-lib-modules\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107040 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5b6d\" (UniqueName: \"kubernetes.io/projected/ecbba608-dfd3-4175-ae83-bf62c402bb80-kube-api-access-k5b6d\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107062 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-sys\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107113 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-podres\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107139 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-sys\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107221 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107207 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-lib-modules\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107465 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107323 2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-proc\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.107465 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.107418 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ecbba608-dfd3-4175-ae83-bf62c402bb80-proc\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.115658 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.115633 2573 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5b6d\" (UniqueName: \"kubernetes.io/projected/ecbba608-dfd3-4175-ae83-bf62c402bb80-kube-api-access-k5b6d\") pod \"perf-node-gather-daemonset-mfnlj\" (UID: \"ecbba608-dfd3-4175-ae83-bf62c402bb80\") " pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.227099 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.227065 2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.362751 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.362606 2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj"] Jun 09 08:25:39.364909 ip-10-0-141-98 kubenswrapper[2573]: W0609 08:25:39.364854 2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podecbba608_dfd3_4175_ae83_bf62c402bb80.slice/crio-8a61c13a7ffd02c8a34d1dedc919ef2cd76175e81d2cacd5d8f36decdbf543ee WatchSource:0}: Error finding container 8a61c13a7ffd02c8a34d1dedc919ef2cd76175e81d2cacd5d8f36decdbf543ee: Status 404 returned error can't find the container with id 8a61c13a7ffd02c8a34d1dedc919ef2cd76175e81d2cacd5d8f36decdbf543ee Jun 09 08:25:39.763067 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.763017 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" event={"ID":"ecbba608-dfd3-4175-ae83-bf62c402bb80","Type":"ContainerStarted","Data":"ec1892838fa7e0ba2cd9364eaf5f1496207792dd22a9e7f9de52e22860352370"} Jun 09 08:25:39.763271 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.763079 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" event={"ID":"ecbba608-dfd3-4175-ae83-bf62c402bb80","Type":"ContainerStarted","Data":"8a61c13a7ffd02c8a34d1dedc919ef2cd76175e81d2cacd5d8f36decdbf543ee"} Jun 09 08:25:39.763271 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.763134 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:39.778646 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:39.778593 2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" podStartSLOduration=1.778575779 podStartE2EDuration="1.778575779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-09 08:25:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-09 08:25:39.777936781 +0000 UTC m=+1637.218400758" watchObservedRunningTime="2026-06-09 08:25:39.778575779 +0000 UTC m=+1637.219039757" Jun 09 08:25:40.128106 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:40.128010 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5v8n5_cd8b2f40-3541-4e14-906b-440cb76c416d/dns/0.log" Jun 09 08:25:40.146243 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:40.146215 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5v8n5_cd8b2f40-3541-4e14-906b-440cb76c416d/kube-rbac-proxy/0.log" Jun 09 08:25:40.248310 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:40.248279 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jc2tq_ed44772a-57dc-425a-8834-266066f4a961/dns-node-resolver/0.log" Jun 09 08:25:40.844770 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:40.844733 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-p89d8_2c562ebb-1fe5-491b-b3de-e5926b9d4cee/node-ca/0.log" Jun 09 08:25:41.876331 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:41.876300 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69895d85c6-x79h4_3bafe8a0-1173-4ad3-afc2-054f00a90e7b/kube-auth-proxy/0.log" Jun 09 08:25:41.973039 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:41.973004 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:41.973039 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:41.973036 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:42.609747 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:42.609689 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jh5j9_b228ca84-f54d-4819-8ab9-bf4b8983ac69/serve-healthcheck-canary/0.log" Jun 09 08:25:43.182368 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:43.182331 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvb4_0042e01a-f1ce-4e6e-be6e-1503350cd4e1/kube-rbac-proxy/0.log" Jun 09 08:25:43.198897 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:43.198852 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvb4_0042e01a-f1ce-4e6e-be6e-1503350cd4e1/exporter/0.log" Jun 09 08:25:43.219199 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:43.219170 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rsvb4_0042e01a-f1ce-4e6e-be6e-1503350cd4e1/extractor/0.log" Jun 09 08:25:45.668414 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:45.668382 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-854649447d-tbcjq_1474308f-4d50-408a-946e-5dbafed9d45c/maas-api/0.log" Jun 09 08:25:45.745124 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:45.745070 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-7f796d5f6-hhxc6_58ee65a8-0ae3-4dae-8891-dda166151679/manager/0.log" Jun 09 08:25:45.778218 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:45.778182 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qzrcm/perf-node-gather-daemonset-mfnlj" Jun 09 08:25:45.932853 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:45.932823 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6b94fd7b-s497m_f4d60053-760c-4d10-b0aa-f08e930c8820/manager/0.log" Jun 09 08:25:45.948689 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:45.948663 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-c6b94fd7b-s497m_f4d60053-760c-4d10-b0aa-f08e930c8820/copy-manifests/0.log" Jun 09 08:25:46.010204 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:46.010176 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-9cprz_5f5b57c4-23fe-4ef1-b3ae-145efe8025cc/postgres/0.log" Jun 09 08:25:47.006412 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:47.006382 2573 scope.go:117] "RemoveContainer" containerID="f057d9b439ee8aa743ec9fe700a203d4877255250640c1087e5ec103b3415014" Jun 09 08:25:47.590920 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:47.590847 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-54ddd4477f-4d4xd_9ecf62de-2a78-4782-bfa7-db90d1ad3e77/manager/0.log" Jun 09 08:25:47.800436 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:47.800350 2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" event={"ID":"ad33c03a-edcf-4141-96cf-f1211fa55f8d","Type":"ContainerStarted","Data":"f66e8856cfe9fee4070b47051c0135f85f004f19a70732fdc6162c50f6793900"} Jun 09 08:25:47.800589 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:47.800565 2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-k57g7" Jun 09 08:25:52.362550 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:52.362508 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69895d85c6-x79h4_3bafe8a0-1173-4ad3-afc2-054f00a90e7b/kube-auth-proxy/0.log" Jun 09 08:25:52.442832 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:52.442796 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/5.log" Jun 09 08:25:52.446293 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:52.446210 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:53.594688 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:53.594658 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69895d85c6-x79h4_3bafe8a0-1173-4ad3-afc2-054f00a90e7b/kube-auth-proxy/0.log" Jun 09 08:25:53.674476 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:53.674437 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/5.log" Jun 09 08:25:53.678327 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:53.678299 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:54.788276 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.788249 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/kube-multus-additional-cni-plugins/0.log" Jun 09 08:25:54.804150 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.804122 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/egress-router-binary-copy/0.log" Jun 09 08:25:54.819831 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.819796 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/cni-plugins/0.log" Jun 09 08:25:54.836546 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.836513 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/bond-cni-plugin/0.log" Jun 09 08:25:54.852786 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.852761 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/routeoverride-cni/0.log" Jun 09 08:25:54.872042 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.872017 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/whereabouts-cni-bincopy/0.log" Jun 09 08:25:54.890205 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.890164 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-wkqzz_8dc53d82-c5ab-4356-ad54-b17575987bc3/whereabouts-cni/0.log" Jun 09 08:25:54.976514 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:54.976479 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mn8fx_7b762bfc-46af-4955-a39b-6e4f4369e856/kube-multus/0.log" Jun 09 08:25:55.085634 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:55.085548 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-h55kx_e1e16526-f4bc-4e20-abb0-999881c8dcc4/network-metrics-daemon/0.log" Jun 09 08:25:55.105986 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:55.105955 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-h55kx_e1e16526-f4bc-4e20-abb0-999881c8dcc4/kube-rbac-proxy/0.log" Jun 09 08:25:55.283162 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:55.283129 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-69895d85c6-x79h4_3bafe8a0-1173-4ad3-afc2-054f00a90e7b/kube-auth-proxy/0.log" Jun 09 08:25:55.347259 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:55.347173 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/5.log" Jun 09 08:25:55.351160 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:55.351135 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-k57g7_ad33c03a-edcf-4141-96cf-f1211fa55f8d/payload-processing/4.log" Jun 09 08:25:56.241626 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.241592 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/ovn-controller/0.log" Jun 09 08:25:56.254878 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.254844 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/ovn-acl-logging/0.log" Jun 09 08:25:56.278996 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.278954 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/kube-rbac-proxy-node/0.log" Jun 09 08:25:56.300659 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.300623 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/kube-rbac-proxy-ovn-metrics/0.log" Jun 09 08:25:56.314636 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.314605 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/northd/0.log" Jun 09 08:25:56.331944 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.331913 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/nbdb/0.log" Jun 09 08:25:56.348168 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.348139 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/sbdb/0.log" Jun 09 08:25:56.411066 ip-10-0-141-98 kubenswrapper[2573]: I0609 08:25:56.411034 2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-ddz24_e3c37a40-2c87-435e-a5e8-6fce48f481c9/ovnkube-controller/0.log"