Jun 08 22:45:15.971188 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 08 22:45:15.971202 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 08 22:45:15.971211 ip-10-0-141-133 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 08 22:45:15.971549 ip-10-0-141-133 systemd[1]: Failed to start Kubernetes Kubelet. Jun 08 22:45:26.038035 ip-10-0-141-133 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 08 22:45:26.038052 ip-10-0-141-133 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 802d7a92e64e457c8259b70bdf0e3d0f -- Jun 08 22:47:58.003828 ip-10-0-141-133 systemd[1]: Starting Kubernetes Kubelet... Jun 08 22:47:58.384256 ip-10-0-141-133 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.384256 ip-10-0-141-133 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 08 22:47:58.384256 ip-10-0-141-133 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.384256 ip-10-0-141-133 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 08 22:47:58.384256 ip-10-0-141-133 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 08 22:47:58.384946 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.384277 2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 08 22:47:58.389436 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389410 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.389436 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389432 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.389436 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389456 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389463 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389470 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389474 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389477 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389479 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389482 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389485 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389488 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389491 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389494 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389496 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389499 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389501 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389504 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389507 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389509 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389513 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389516 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.389625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389518 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389522 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389526 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389529 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389532 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389534 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389537 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389540 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389542 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389545 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389547 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389550 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389552 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389555 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389557 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389562 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389564 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389568 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389570 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389573 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.390081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389575 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389578 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389580 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389583 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389586 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389588 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389591 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389593 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389596 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389599 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389601 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389604 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389607 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389609 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389611 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389614 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389616 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389619 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389622 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389625 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.390606 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389627 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389630 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389632 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389635 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389638 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389640 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389643 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389646 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389650 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389652 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389655 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389659 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389661 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389664 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389666 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389669 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389672 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389674 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389677 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.391203 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389679 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389681 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389685 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389689 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389693 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389696 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389699 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389702 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389705 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389707 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389710 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389712 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389714 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389717 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389719 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389722 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389724 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389727 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.389729 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.391695 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390920 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390930 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390934 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390936 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390940 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390943 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390946 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390949 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390952 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390955 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390958 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390961 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390964 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390967 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390969 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390973 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390977 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390980 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390983 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390986 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.392163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390989 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390992 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390995 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.390997 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391000 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391002 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391005 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391007 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391009 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391012 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391014 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391017 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391020 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391023 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391027 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391029 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391032 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391035 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391037 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391039 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.392679 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391042 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391045 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391047 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391049 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391052 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391054 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391057 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391059 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391062 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391064 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391068 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391070 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391072 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391075 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391078 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391080 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391082 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391085 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391088 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391092 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.393177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391096 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391099 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391102 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391106 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391109 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391112 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391114 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391117 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391120 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391122 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391125 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391127 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391130 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391133 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391136 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391139 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391142 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391145 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391147 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.393687 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391150 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391153 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391156 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391159 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391163 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391166 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391169 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391171 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391174 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391177 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391179 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391182 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391184 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391187 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391189 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391192 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391194 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391197 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391200 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.394163 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391202 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391608 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391613 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391616 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391619 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391621 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391624 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391627 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391631 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391633 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391636 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391638 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391642 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391645 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391647 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391650 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391652 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391657 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391661 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391664 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.394683 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391666 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391669 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391672 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391675 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391677 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391680 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391683 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391686 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391690 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391693 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391696 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391700 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391702 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391705 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391708 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391711 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391714 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391716 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391719 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.395165 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391723 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391726 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391728 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391731 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391734 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391736 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391739 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391742 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391745 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391748 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391750 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391753 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391756 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391758 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391761 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391763 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391766 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391769 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391771 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391774 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.395664 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391776 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391779 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391781 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391784 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391787 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391790 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391792 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391795 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391797 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391800 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391802 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391806 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391809 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391812 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391815 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391817 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391820 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391822 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391824 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.396193 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391827 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391829 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391832 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391835 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391837 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391841 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391845 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391848 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391851 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391853 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391856 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391858 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391861 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391863 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391865 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391870 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391873 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391876 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391879 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391881 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.396677 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391884 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.391886 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391963 2583 flags.go:64] FLAG: --address="0.0.0.0" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391970 2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391978 2583 flags.go:64] FLAG: --anonymous-auth="true" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391982 2583 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391986 2583 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391990 2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391994 2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.391998 2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392001 2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392004 2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392008 2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392011 2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392014 2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392017 2583 flags.go:64] FLAG: --cgroup-root="" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392020 2583 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392023 2583 flags.go:64] FLAG: --client-ca-file="" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392036 2583 flags.go:64] FLAG: --cloud-provider="external" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392039 2583 flags.go:64] FLAG: --cluster-dns="[]" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392043 2583 flags.go:64] FLAG: --cluster-domain="" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392046 2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392049 2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392052 2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 08 22:47:58.397169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392055 2583 flags.go:64] FLAG: --container-log-max-files="5" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392064 2583 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392067 2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392074 2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392078 2583 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392081 2583 flags.go:64] FLAG: --contention-profiling="false" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392084 2583 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392087 2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392090 2583 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392093 2583 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392097 2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392100 2583 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392103 2583 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392106 2583 flags.go:64] FLAG: --enable-load-reader="false" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392109 2583 flags.go:64] FLAG: --enable-server="true" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392111 2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392117 2583 flags.go:64] FLAG: --event-burst="100" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392120 2583 flags.go:64] FLAG: --event-qps="50" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392123 2583 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392126 2583 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392129 2583 flags.go:64] FLAG: --eviction-hard="" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392132 2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392135 2583 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392138 2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392141 2583 flags.go:64] FLAG: --eviction-soft="" Jun 08 22:47:58.397789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392144 2583 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392147 2583 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392153 2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392156 2583 flags.go:64] FLAG: --experimental-mounter-path="" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392159 2583 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392162 2583 flags.go:64] FLAG: --fail-swap-on="true" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392165 2583 flags.go:64] FLAG: --feature-gates="" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392169 2583 flags.go:64] FLAG: --file-check-frequency="20s" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392172 2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392175 2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392180 2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392183 2583 flags.go:64] FLAG: --healthz-port="10248" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392185 2583 flags.go:64] FLAG: --help="false" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392189 2583 flags.go:64] FLAG: --hostname-override="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392192 2583 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392195 2583 flags.go:64] FLAG: --http-check-frequency="20s" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392198 2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392201 2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392204 2583 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392207 2583 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392211 2583 flags.go:64] FLAG: --image-service-endpoint="" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392214 2583 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392217 2583 flags.go:64] FLAG: --kube-api-burst="100" Jun 08 22:47:58.398417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392219 2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392222 2583 flags.go:64] FLAG: --kube-api-qps="50" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392225 2583 flags.go:64] FLAG: --kube-reserved="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392228 2583 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392231 2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392234 2583 flags.go:64] FLAG: --kubelet-cgroups="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392236 2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392239 2583 flags.go:64] FLAG: --lock-file="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392242 2583 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392244 2583 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392247 2583 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392252 2583 flags.go:64] FLAG: --log-json-split-stream="false" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392256 2583 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392259 2583 flags.go:64] FLAG: --log-text-split-stream="false" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392262 2583 flags.go:64] FLAG: --logging-format="text" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392265 2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392268 2583 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392270 2583 flags.go:64] FLAG: --manifest-url="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392273 2583 flags.go:64] FLAG: --manifest-url-header="" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392279 2583 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392282 2583 flags.go:64] FLAG: --max-open-files="1000000" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392286 2583 flags.go:64] FLAG: --max-pods="110" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392289 2583 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392292 2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392295 2583 flags.go:64] FLAG: --memory-manager-policy="None" Jun 08 22:47:58.399100 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392298 2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392301 2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392304 2583 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392307 2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392317 2583 flags.go:64] FLAG: --node-status-max-images="50" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392320 2583 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392323 2583 flags.go:64] FLAG: --oom-score-adj="-999" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392326 2583 flags.go:64] FLAG: --pod-cidr="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392329 2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392335 2583 flags.go:64] FLAG: --pod-manifest-path="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392337 2583 flags.go:64] FLAG: --pod-max-pids="-1" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392340 2583 flags.go:64] FLAG: --pods-per-core="0" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392343 2583 flags.go:64] FLAG: --port="10250" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392346 2583 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392349 2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0a22790cfcc874895" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392352 2583 flags.go:64] FLAG: --qos-reserved="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392355 2583 flags.go:64] FLAG: --read-only-port="10255" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392358 2583 flags.go:64] FLAG: --register-node="true" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392361 2583 flags.go:64] FLAG: --register-with-taints="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392365 2583 flags.go:64] FLAG: --registry-burst="10" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392368 2583 flags.go:64] FLAG: --registry-qps="5" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392371 2583 flags.go:64] FLAG: --reserved-cpus="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392374 2583 flags.go:64] FLAG: --reserved-memory="" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392377 2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392380 2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 08 22:47:58.399738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392383 2583 flags.go:64] FLAG: --rotate-certificates="false" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392386 2583 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392389 2583 flags.go:64] FLAG: --runonce="false" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392392 2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392395 2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392398 2583 flags.go:64] FLAG: --seccomp-default="false" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392401 2583 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392403 2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392406 2583 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392409 2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392412 2583 flags.go:64] FLAG: --storage-driver-password="root" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392416 2583 flags.go:64] FLAG: --storage-driver-secure="false" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392419 2583 flags.go:64] FLAG: --storage-driver-table="stats" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392422 2583 flags.go:64] FLAG: --storage-driver-user="root" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392424 2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392427 2583 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392431 2583 flags.go:64] FLAG: --system-cgroups="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392434 2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392454 2583 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392457 2583 flags.go:64] FLAG: --tls-cert-file="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392460 2583 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392464 2583 flags.go:64] FLAG: --tls-min-version="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392467 2583 flags.go:64] FLAG: --tls-private-key-file="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392469 2583 flags.go:64] FLAG: --topology-manager-policy="none" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392472 2583 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 08 22:47:58.400343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392475 2583 flags.go:64] FLAG: --topology-manager-scope="container" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392478 2583 flags.go:64] FLAG: --v="2" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392482 2583 flags.go:64] FLAG: --version="false" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392487 2583 flags.go:64] FLAG: --vmodule="" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392490 2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.392494 2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392630 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392634 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392637 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392641 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392644 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392646 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392649 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392651 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392654 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392657 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392659 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392662 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392667 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392671 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.400984 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392673 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392676 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392680 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392683 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392685 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392688 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392691 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392694 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392696 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392699 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392701 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392704 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392706 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392709 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392711 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392714 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392717 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392721 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392724 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392727 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.401491 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392729 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392733 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392736 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392738 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392741 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392744 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392746 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392749 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392752 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392754 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392758 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392761 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392763 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392766 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392768 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392771 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392773 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392776 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392778 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392781 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.401993 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392783 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392786 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392789 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392791 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392793 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392796 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392799 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392801 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392804 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392807 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392809 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392812 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392814 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392818 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392820 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392823 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392825 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392828 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392830 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.402520 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392833 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392836 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392838 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392842 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392845 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392847 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392850 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392852 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392855 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392858 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392860 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392862 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392865 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392868 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392870 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392872 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392876 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392879 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392882 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.402980 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392885 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392888 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392891 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392893 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392896 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392899 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.392901 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.393406 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.400153 2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.400271 2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.400291 2583 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.400298 2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400354 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400359 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400362 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.403548 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400365 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400368 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400371 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400375 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400377 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400380 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400383 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400386 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400388 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400391 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400393 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400396 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400399 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400402 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400404 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400407 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400409 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400412 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400414 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400416 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.403957 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400419 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400422 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400424 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400426 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400429 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400432 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400450 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400453 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400456 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400459 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400462 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400464 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400467 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400470 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400472 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400475 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400477 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400480 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400482 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400484 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.404468 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400487 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400489 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400492 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400494 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400498 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400502 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400506 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400508 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400511 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400514 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400516 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400519 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400521 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400524 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400527 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400529 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400532 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400535 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400538 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400901 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.405072 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400904 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400907 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400910 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400912 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400915 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400917 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400921 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400925 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400928 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400931 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400934 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400936 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400939 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400942 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400944 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400947 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400949 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400953 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400955 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.405589 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400958 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400960 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400963 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400966 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400970 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400972 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400975 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400978 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400980 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400983 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400985 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400989 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400991 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400995 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.400998 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401001 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.406109 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401003 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.401009 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401125 2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401131 2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401134 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401137 2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401140 2583 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401143 2583 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401145 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401148 2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401150 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401153 2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401156 2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401158 2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401161 2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 08 22:47:58.406914 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401163 2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401166 2583 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401168 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401171 2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401173 2583 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401176 2583 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401178 2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401181 2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401183 2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401185 2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401188 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401190 2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401193 2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401195 2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401198 2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401202 2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401206 2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401209 2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401212 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 08 22:47:58.407481 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401214 2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401217 2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401220 2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401223 2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401226 2583 feature_gate.go:328] unrecognized feature gate: Example Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401228 2583 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401231 2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401233 2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401235 2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401238 2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401241 2583 feature_gate.go:328] unrecognized feature gate: Example2 Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401244 2583 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401246 2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401249 2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401251 2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401254 2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401256 2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401259 2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401262 2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401264 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 08 22:47:58.408002 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401267 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401269 2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401272 2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401274 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401277 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401279 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401282 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401284 2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401286 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401289 2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401292 2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401294 2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401297 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401299 2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401302 2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401304 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401306 2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401309 2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401311 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401314 2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 08 22:47:58.408926 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401316 2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401319 2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401321 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401324 2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401327 2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401329 2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401333 2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401337 2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401341 2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401343 2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401346 2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401349 2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401352 2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401355 2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401357 2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401360 2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401363 2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401366 2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401369 2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 08 22:47:58.409430 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401371 2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401374 2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401377 2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401382 2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401385 2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401387 2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401390 2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:58.401393 2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.401398 2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.401536 2583 server.go:962] "Client rotation is on, will bootstrap in background" Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.403392 2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.404642 2583 server.go:1019] "Starting client certificate rotation" Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.404757 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 22:47:58.409950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.404812 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 08 22:47:58.410292 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.406914 2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 22:47:58.410292 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.408469 2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 08 22:47:58.424363 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.424340 2583 log.go:25] "Validated CRI v1 runtime API" Jun 08 22:47:58.426820 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.426803 2583 log.go:25] "Validated CRI v1 image API" Jun 08 22:47:58.429412 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.429391 2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 08 22:47:58.433640 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.433614 2583 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b4bf0af3-c5d2-40fe-9f91-9c8508346140:/dev/nvme0n1p4 d408e742-5f7d-4f7a-8c12-01a97951915c:/dev/nvme0n1p3] Jun 08 22:47:58.433735 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.433637 2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 08 22:47:58.438582 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.438458 2583 manager.go:216] Machine: {Timestamp:2026-06-08 22:47:58.437399226 +0000 UTC m=+0.317306310 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3125672 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2c3104cf9c23763034a513696637a8 SystemUUID:ec2c3104-cf9c-2376-3034-a513696637a8 BootID:802d7a92-e64e-457c-8259-b70bdf0e3d0f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:c0:b6:c3:21:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:c0:b6:c3:21:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8a:4f:b9:61:c5:08 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 08 22:47:58.438582 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.438572 2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 08 22:47:58.438737 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.438664 2583 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 08 22:47:58.441309 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.441275 2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 08 22:47:58.441476 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.441313 2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-133.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 08 22:47:58.441963 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.441952 2583 topology_manager.go:138] "Creating topology manager with none policy" Jun 08 22:47:58.441999 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.441966 2583 container_manager_linux.go:309] "Creating device plugin manager" Jun 08 22:47:58.441999 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.441974 2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 22:47:58.442338 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.442320 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 22:47:58.442865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.442851 2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 08 22:47:58.443571 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.443561 2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 08 22:47:58.445835 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.445824 2583 state_mem.go:36] "Initialized new in-memory state store" Jun 08 22:47:58.445969 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.445959 2583 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 08 22:47:58.446009 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.445991 2583 kubelet.go:486] "Attempting to sync node with API server" Jun 08 22:47:58.446009 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446002 2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 08 22:47:58.446082 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446023 2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 08 22:47:58.446082 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446034 2583 kubelet.go:398] "Adding apiserver pod source" Jun 08 22:47:58.446082 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446047 2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 08 22:47:58.446525 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446510 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 22:47:58.446573 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.446538 2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 08 22:47:58.450629 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.450605 2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 08 22:47:58.452485 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.452466 2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 08 22:47:58.452588 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.452499 2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 08 22:47:58.453404 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453391 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 08 22:47:58.453404 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453407 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453413 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453419 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453429 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453435 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453463 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453469 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453477 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453483 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453491 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 08 22:47:58.453536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.453498 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 08 22:47:58.454111 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.454101 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 08 22:47:58.454153 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.454114 2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 08 22:47:58.457697 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.457681 2583 server.go:1268] "Started kubelet" Jun 08 22:47:58.457838 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.457765 2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 08 22:47:58.458360 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.457900 2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 08 22:47:58.458360 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.458007 2583 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 08 22:47:58.458726 ip-10-0-141-133 systemd[1]: Started Kubernetes Kubelet. Jun 08 22:47:58.459962 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.459942 2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 08 22:47:58.460274 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.460257 2583 server.go:310] "Adding debug handlers to kubelet server" Jun 08 22:47:58.460528 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.460509 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5xg22" Jun 08 22:47:58.461238 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.461211 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 08 22:47:58.462249 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.462221 2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-133.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 08 22:47:58.462327 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.462279 2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-133.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 08 22:47:58.464636 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.464613 2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 08 22:47:58.465659 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.464629 2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-133.ec2.internal.18b73ca9fe0bce45 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-133.ec2.internal,UID:ip-10-0-141-133.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-133.ec2.internal,},FirstTimestamp:2026-06-08 22:47:58.457646661 +0000 UTC m=+0.337553747,LastTimestamp:2026-06-08 22:47:58.457646661 +0000 UTC m=+0.337553747,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-133.ec2.internal,}" Jun 08 22:47:58.466234 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.466206 2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 08 22:47:58.466234 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.466222 2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 08 22:47:58.466584 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.466430 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.466857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.466836 2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 08 22:47:58.466966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.466946 2583 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 08 22:47:58.466966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.466968 2583 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 08 22:47:58.467119 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467053 2583 factory.go:55] Registering systemd factory Jun 08 22:47:58.467119 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467066 2583 factory.go:223] Registration of the systemd container factory successfully Jun 08 22:47:58.467119 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467095 2583 reconstruct.go:97] "Volume reconstruction finished" Jun 08 22:47:58.467119 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467107 2583 reconciler.go:29] "Reconciler: start to sync state" Jun 08 22:47:58.467399 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467378 2583 factory.go:153] Registering CRI-O factory Jun 08 22:47:58.467399 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467401 2583 factory.go:223] Registration of the crio container factory successfully Jun 08 22:47:58.467539 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467466 2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 08 22:47:58.467539 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467491 2583 factory.go:103] Registering Raw factory Jun 08 22:47:58.467539 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467505 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5xg22" Jun 08 22:47:58.467539 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467509 2583 manager.go:1196] Started watching for new ooms in manager Jun 08 22:47:58.467929 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.467917 2583 manager.go:349] Starting recovery of all containers Jun 08 22:47:58.471729 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.471708 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:47:58.474085 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.474056 2583 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.478792 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.478712 2583 manager.go:354] Recovery completed Jun 08 22:47:58.480281 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.480256 2583 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jun 08 22:47:58.483488 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.483474 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.485562 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.485543 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.485634 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.485575 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.485634 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.485593 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.486182 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486166 2583 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 08 22:47:58.486267 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486182 2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 08 22:47:58.486267 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486171 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.486267 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486203 2583 state_mem.go:36] "Initialized new in-memory state store" Jun 08 22:47:58.486928 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486914 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.486978 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486951 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.486978 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.486960 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.487264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.487252 2583 policy_none.go:49] "None policy: Start" Jun 08 22:47:58.487323 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.487272 2583 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 08 22:47:58.487323 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.487286 2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 08 22:47:58.488154 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.488140 2583 policy_none.go:47] "Start" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536112 2583 manager.go:337] "Starting Device Plugin manager" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.536154 2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536166 2583 server.go:87] "Starting device plugin registration server" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536572 2583 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536602 2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536812 2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536903 2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.536913 2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.537024 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.537896 2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.537940 2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.538825 2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.538869 2583 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.538891 2583 kubelet.go:2448] "Starting kubelet main sync loop" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.538932 2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 08 22:47:58.542890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.540636 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:47:58.637484 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.637376 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.638481 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.638462 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.638580 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.638501 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.638580 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.638515 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.638580 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.638548 2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.639531 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.639482 2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal"] Jun 08 22:47:58.639597 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.639580 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.640335 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.640319 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.640418 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.640353 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.640418 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.640366 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.642724 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.642708 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.642832 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.642815 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.642880 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.642851 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.643563 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643544 2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.643658 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.643567 2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-133.ec2.internal\": node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.643724 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643687 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.643724 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643714 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.643724 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643723 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.643817 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643691 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.643817 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643806 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.643877 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.643818 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.645997 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.645981 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.646072 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.646009 2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 08 22:47:58.646786 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.646768 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientMemory" Jun 08 22:47:58.646876 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.646799 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasNoDiskPressure" Jun 08 22:47:58.646876 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.646811 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeHasSufficientPID" Jun 08 22:47:58.654779 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.654762 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.663104 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.663081 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.667221 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.667200 2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-133.ec2.internal\" not found" node="ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.668499 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.668478 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.668553 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.668508 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.668553 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.668527 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7d005ddae688368c7345a44445a431e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"b7d005ddae688368c7345a44445a431e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.755866 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.755833 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.769120 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769092 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.769120 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.769345 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769147 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7d005ddae688368c7345a44445a431e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"b7d005ddae688368c7345a44445a431e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.769345 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769185 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b7d005ddae688368c7345a44445a431e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-133.ec2.internal\" (UID: \"b7d005ddae688368c7345a44445a431e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.769345 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769207 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.769345 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.769204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41cd31e5c82cbe87a53c1c85e967347a-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal\" (UID: \"41cd31e5c82cbe87a53c1c85e967347a\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.856240 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.856195 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.956994 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:58.956934 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:58.969190 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.969164 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:47:58.974250 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:58.973931 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:47:59.057714 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.057674 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.158282 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.158247 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.258848 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.258757 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.359277 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.359244 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.404786 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.404745 2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 08 22:47:59.405521 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.404944 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 22:47:59.405521 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.404956 2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 08 22:47:59.460223 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.460185 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.470743 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.470703 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-07 22:42:58 +0000 UTC" deadline="2028-03-25 12:34:10.54139814 +0000 UTC" Jun 08 22:47:59.470743 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.470739 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15733h46m11.070663381s" Jun 08 22:47:59.542822 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:59.542791 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7d005ddae688368c7345a44445a431e.slice/crio-fa58b458037c212cb921f35ca73d800c3a90555ecce066057c4fc7b70d347678 WatchSource:0}: Error finding container fa58b458037c212cb921f35ca73d800c3a90555ecce066057c4fc7b70d347678: Status 404 returned error can't find the container with id fa58b458037c212cb921f35ca73d800c3a90555ecce066057c4fc7b70d347678 Jun 08 22:47:59.543347 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:47:59.543327 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41cd31e5c82cbe87a53c1c85e967347a.slice/crio-f5578c0be729d9c5f06739e3878297d0adcdc79fba52c4c1154d1f3112ae0b31 WatchSource:0}: Error finding container f5578c0be729d9c5f06739e3878297d0adcdc79fba52c4c1154d1f3112ae0b31: Status 404 returned error can't find the container with id f5578c0be729d9c5f06739e3878297d0adcdc79fba52c4c1154d1f3112ae0b31 Jun 08 22:47:59.547297 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.547274 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 22:47:59.560431 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.560395 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.660981 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.660937 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.761542 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.761485 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.862161 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.862059 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.962968 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:47:59.962931 2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-133.ec2.internal\" not found" Jun 08 22:47:59.983118 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:47:59.983083 2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.064200 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.064169 2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.067215 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.067180 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" Jun 08 22:48:00.076944 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.076917 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 22:48:00.078215 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.078184 2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" Jun 08 22:48:00.085739 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.085716 2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 08 22:48:00.306175 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.306098 2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.448053 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.448017 2583 apiserver.go:52] "Watching apiserver" Jun 08 22:48:00.453027 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.452995 2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 08 22:48:00.454748 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.454727 2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 08 22:48:00.456279 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.456251 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-rlvhw","kube-system/global-pull-secret-syncer-x7rkj","kube-system/konnectivity-agent-htb6v","kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal","openshift-multus/multus-bj7xh","openshift-multus/network-metrics-daemon-lw8qj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx","openshift-cluster-node-tuning-operator/tuned-mcsjw","openshift-dns/node-resolver-vk4sh","openshift-image-registry/node-ca-lcf6r","openshift-multus/multus-additional-cni-plugins-7t5hd","openshift-network-diagnostics/network-check-target-fjz6d","openshift-network-operator/iptables-alerter-hb47l"] Jun 08 22:48:00.459078 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.459058 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.461103 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.461082 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.461344 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.461325 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2s8nc\"" Jun 08 22:48:00.461660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.461641 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.461801 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.461790 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.463717 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.463227 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.463717 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.463604 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-62gb9\"" Jun 08 22:48:00.463717 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.463610 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.463906 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.463863 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 08 22:48:00.463949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.463921 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.465318 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.465295 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.466377 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.466356 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 08 22:48:00.466848 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.466831 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-nkkls\"" Jun 08 22:48:00.467856 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.467835 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 08 22:48:00.468553 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.468459 2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 08 22:48:00.468553 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.468501 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.470918 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.470878 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 08 22:48:00.470918 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.470895 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.471057 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.470976 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8ccx2\"" Jun 08 22:48:00.472081 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472059 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.472253 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472235 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-wftb4\"" Jun 08 22:48:00.472419 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472402 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.472633 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472611 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 08 22:48:00.472946 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472917 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.473052 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.472984 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.475215 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475162 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.475390 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475370 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.475526 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475507 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-r6xx4\"" Jun 08 22:48:00.475716 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475700 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.475765 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475749 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 08 22:48:00.475881 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.475866 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 08 22:48:00.477638 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.477618 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:00.477727 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.477688 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:00.478021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.478003 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.478093 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.478078 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-8cnvg\"" Jun 08 22:48:00.478272 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.478258 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479744 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b52ada8-b170-4037-a664-fdd46e23204c-agent-certs\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479784 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-device-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479813 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-daemon-config\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479826 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479837 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-hosts-file\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479863 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-tmp-dir\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.479890 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479954 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/233ee4f5-e859-436a-966f-db64a39f02ba-host-slash\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.479991 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbsmm\" (UniqueName: \"kubernetes.io/projected/233ee4f5-e859-436a-966f-db64a39f02ba-kube-api-access-zbsmm\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.480029 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480014 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-os-release\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-socket-dir-parent\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480088 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-bin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480113 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-multus\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480138 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27mg4\" (UniqueName: \"kubernetes.io/projected/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-kube-api-access-27mg4\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480175 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7948a206-759b-4031-a1bc-af2b2e94ec55-host\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480199 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-socket-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480237 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-system-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480242 2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480263 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-hostroot\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480287 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-multus-certs\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-registration-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480342 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-conf-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480364 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/233ee4f5-e859-436a-966f-db64a39f02ba-iptables-alerter-script\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480395 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-sys-fs\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480416 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-kubelet\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480457 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b52ada8-b170-4037-a664-fdd46e23204c-konnectivity-ca\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.480547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480475 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480490 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-netns\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480506 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kf8c\" (UniqueName: \"kubernetes.io/projected/5c74d00f-6de3-434f-8975-abd5eeda6c8e-kube-api-access-9kf8c\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480520 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7948a206-759b-4031-a1bc-af2b2e94ec55-serviceca\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480535 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7s2\" (UniqueName: \"kubernetes.io/projected/7948a206-759b-4031-a1bc-af2b2e94ec55-kube-api-access-pk7s2\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480561 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrsf6\" (UniqueName: \"kubernetes.io/projected/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kube-api-access-hrsf6\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480576 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-etc-kubernetes\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480592 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480608 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480630 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cnibin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480644 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cni-binary-copy\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.481212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.480659 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-k8s-cni-cncf-io\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.482211 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.482183 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.484960 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.484937 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bvtwd\"" Jun 08 22:48:00.485054 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485012 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 08 22:48:00.485145 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485126 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 08 22:48:00.485145 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485137 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 08 22:48:00.485270 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485144 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 08 22:48:00.485270 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485179 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 08 22:48:00.485339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485296 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 08 22:48:00.485339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.485312 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.485405 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.485380 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:00.487156 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.486981 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.489241 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.489222 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 08 22:48:00.489539 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.489518 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-nmb2j\"" Jun 08 22:48:00.489626 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.489523 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 08 22:48:00.543462 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.543395 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"b7d005ddae688368c7345a44445a431e","Type":"ContainerStarted","Data":"fa58b458037c212cb921f35ca73d800c3a90555ecce066057c4fc7b70d347678"} Jun 08 22:48:00.544602 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.544561 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"41cd31e5c82cbe87a53c1c85e967347a","Type":"ContainerStarted","Data":"f5578c0be729d9c5f06739e3878297d0adcdc79fba52c4c1154d1f3112ae0b31"} Jun 08 22:48:00.568557 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.568480 2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 08 22:48:00.581650 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-multus-certs\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.581811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581660 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-kubernetes\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.581811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581684 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-tuned\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.581811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581719 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-multus-certs\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.581811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581744 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-conf-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.581811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581787 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-systemd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581830 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-conf-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581809 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-log-socket\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581877 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581912 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-bin\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581932 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581959 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/233ee4f5-e859-436a-966f-db64a39f02ba-iptables-alerter-script\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.581991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.581988 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-kubelet-config\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582004 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-run\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582023 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-lib-modules\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582038 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v57bj\" (UniqueName: \"kubernetes.io/projected/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-kube-api-access-v57bj\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582054 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jmwb\" (UniqueName: \"kubernetes.io/projected/dd770a03-362b-4433-bbe6-50a73e3c683e-kube-api-access-4jmwb\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b52ada8-b170-4037-a664-fdd46e23204c-konnectivity-ca\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582100 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582116 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582132 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582148 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:00.582218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582163 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582228 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-node-log\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582244 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-systemd\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582258 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-tmp\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582273 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582287 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582305 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cnibin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582319 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-config\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582341 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b52ada8-b170-4037-a664-fdd46e23204c-agent-certs\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582363 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-daemon-config\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582381 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-hosts-file\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582397 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovn-node-metrics-cert\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582489 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysconfig\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582505 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-system-cni-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbsmm\" (UniqueName: \"kubernetes.io/projected/233ee4f5-e859-436a-966f-db64a39f02ba-kube-api-access-zbsmm\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582539 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-socket-dir-parent\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582554 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.582660 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582570 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-env-overrides\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582589 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-host\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582604 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-dbus\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582631 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582646 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-cnibin\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582661 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-registration-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582676 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-var-lib-kubelet\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582693 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hhq\" (UniqueName: \"kubernetes.io/projected/e55c9f37-cba5-4014-a612-33d35b24fa8d-kube-api-access-t2hhq\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582712 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-os-release\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-sys-fs\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582748 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-kubelet\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582764 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-slash\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582779 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-script-lib\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-conf\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582819 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582836 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-netns\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583183 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kf8c\" (UniqueName: \"kubernetes.io/projected/5c74d00f-6de3-434f-8975-abd5eeda6c8e-kube-api-access-9kf8c\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582945 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-systemd-units\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582959 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-ovn\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582976 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7948a206-759b-4031-a1bc-af2b2e94ec55-serviceca\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.582991 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7s2\" (UniqueName: \"kubernetes.io/projected/7948a206-759b-4031-a1bc-af2b2e94ec55-kube-api-access-pk7s2\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583007 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrsf6\" (UniqueName: \"kubernetes.io/projected/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kube-api-access-hrsf6\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583021 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-etc-kubernetes\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583036 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-netd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583051 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-kube-api-access-gqsfk\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583067 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cni-binary-copy\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583083 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-k8s-cni-cncf-io\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583097 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-netns\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583111 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-var-lib-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583126 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-etc-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583143 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-sys\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583159 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-device-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583176 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-tmp-dir\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.583779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583193 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-kubelet\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583209 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/233ee4f5-e859-436a-966f-db64a39f02ba-host-slash\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583241 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-os-release\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583258 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-bin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-multus\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583293 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27mg4\" (UniqueName: \"kubernetes.io/projected/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-kube-api-access-27mg4\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583314 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-modprobe-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583333 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583349 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7948a206-759b-4031-a1bc-af2b2e94ec55-host\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583365 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-socket-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583379 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-system-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-hostroot\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583473 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-hostroot\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583497 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-netns\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583692 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-kubelet\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583783 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-system-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583810 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-etc-kubernetes\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.584545 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583849 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-hosts-file\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583889 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-run-k8s-cni-cncf-io\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583946 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-os-release\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.583993 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-device-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584079 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584134 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-cni-dir\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584162 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/233ee4f5-e859-436a-966f-db64a39f02ba-iptables-alerter-script\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584204 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7948a206-759b-4031-a1bc-af2b2e94ec55-serviceca\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584171 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/233ee4f5-e859-436a-966f-db64a39f02ba-host-slash\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584216 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cnibin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584218 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-registration-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584244 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7948a206-759b-4031-a1bc-af2b2e94ec55-host\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584260 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-bin\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584266 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-host-var-lib-cni-multus\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584295 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-sys-fs\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584308 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-etc-selinux\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584363 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-socket-dir-parent\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584364 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-cni-binary-copy\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584370 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/5c74d00f-6de3-434f-8975-abd5eeda6c8e-multus-daemon-config\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.585857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584466 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-socket-dir\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.585857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584603 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-tmp-dir\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.585857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584758 2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 08 22:48:00.585857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.584853 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b52ada8-b170-4037-a664-fdd46e23204c-konnectivity-ca\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.587887 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.587867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b52ada8-b170-4037-a664-fdd46e23204c-agent-certs\") pod \"konnectivity-agent-htb6v\" (UID: \"4b52ada8-b170-4037-a664-fdd46e23204c\") " pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.596938 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.596908 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7s2\" (UniqueName: \"kubernetes.io/projected/7948a206-759b-4031-a1bc-af2b2e94ec55-kube-api-access-pk7s2\") pod \"node-ca-lcf6r\" (UID: \"7948a206-759b-4031-a1bc-af2b2e94ec55\") " pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.597459 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.597413 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrsf6\" (UniqueName: \"kubernetes.io/projected/6f666e24-022d-4c5b-b09d-be5ecd0b4c77-kube-api-access-hrsf6\") pod \"aws-ebs-csi-driver-node-xsqzx\" (UID: \"6f666e24-022d-4c5b-b09d-be5ecd0b4c77\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.597949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.597925 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-27mg4\" (UniqueName: \"kubernetes.io/projected/d70d3927-89dd-457d-a77c-2d4c4bdaf48a-kube-api-access-27mg4\") pod \"node-resolver-vk4sh\" (UID: \"d70d3927-89dd-457d-a77c-2d4c4bdaf48a\") " pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.598339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.598317 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbsmm\" (UniqueName: \"kubernetes.io/projected/233ee4f5-e859-436a-966f-db64a39f02ba-kube-api-access-zbsmm\") pod \"iptables-alerter-hb47l\" (UID: \"233ee4f5-e859-436a-966f-db64a39f02ba\") " pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.598339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.598331 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kf8c\" (UniqueName: \"kubernetes.io/projected/5c74d00f-6de3-434f-8975-abd5eeda6c8e-kube-api-access-9kf8c\") pod \"multus-bj7xh\" (UID: \"5c74d00f-6de3-434f-8975-abd5eeda6c8e\") " pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.683711 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683670 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-systemd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683711 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683717 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-log-socket\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683746 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683792 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-log-socket\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683796 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-systemd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683823 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-bin\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683853 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683871 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-kubelet-config\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683901 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-bin\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683928 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-kubelet-config\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.683949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.683935 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-run\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.684007 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684042 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-run\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684066 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-lib-modules\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.684092 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.184067351 +0000 UTC m=+3.063974441 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684118 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v57bj\" (UniqueName: \"kubernetes.io/projected/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-kube-api-access-v57bj\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684141 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jmwb\" (UniqueName: \"kubernetes.io/projected/dd770a03-362b-4433-bbe6-50a73e3c683e-kube-api-access-4jmwb\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684178 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-lib-modules\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684412 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684430 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684477 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684519 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-node-log\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684537 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-systemd\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684556 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-tmp\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684580 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684581 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-config\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684621 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovn-node-metrics-cert\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684644 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysconfig\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684668 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-system-cni-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684696 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684720 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-env-overrides\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684741 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-host\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684762 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-dbus\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684786 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684810 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-cnibin\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.684935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684833 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-var-lib-kubelet\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684856 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hhq\" (UniqueName: \"kubernetes.io/projected/e55c9f37-cba5-4014-a612-33d35b24fa8d-kube-api-access-t2hhq\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684879 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-os-release\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684901 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-slash\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684922 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-script-lib\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-conf\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684969 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.684998 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-systemd-units\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685023 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-ovn\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685054 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-netd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685076 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-kube-api-access-gqsfk\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685105 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-netns\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685127 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-var-lib-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-etc-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685188 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-sys\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685216 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-kubelet\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685242 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.685727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685276 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-modprobe-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685312 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685353 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-kubernetes\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685356 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-binary-copy\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685376 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-tuned\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685416 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-node-log\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685527 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-systemd\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685814 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-etc-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685843 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysctl-conf\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-modprobe-d\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685959 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-sys\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685967 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.685988 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-kubelet\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686015 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-systemd-units\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686057 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-run-ovn\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686097 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-cni-netd\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686116 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-config\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686211 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-run-netns\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.686331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686226 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-var-lib-openvswitch\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686218 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-kubernetes\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686295 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-sysconfig\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686336 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-cnibin\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686359 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-system-cni-dir\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686391 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-var-lib-kubelet\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686400 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686417 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686459 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-host-slash\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686533 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd770a03-362b-4433-bbe6-50a73e3c683e-os-release\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686528 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e55c9f37-cba5-4014-a612-33d35b24fa8d-host\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.686606 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686632 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1892e47-8908-4673-b782-e55723903d13-dbus\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.686712 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.186691473 +0000 UTC m=+3.066598559 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686834 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-env-overrides\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.687096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.686885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd770a03-362b-4433-bbe6-50a73e3c683e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.687937 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.687909 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovnkube-script-lib\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.688907 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.688885 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-etc-tuned\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.689135 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.689115 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-ovn-node-metrics-cert\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.689256 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.689243 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e55c9f37-cba5-4014-a612-33d35b24fa8d-tmp\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.690719 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.690696 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:00.690841 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.690723 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:00.690841 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.690738 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:00.690910 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:00.690854 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:01.19083603 +0000 UTC m=+3.070743101 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:00.692473 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.692433 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v57bj\" (UniqueName: \"kubernetes.io/projected/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-kube-api-access-v57bj\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:00.692646 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.692624 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jmwb\" (UniqueName: \"kubernetes.io/projected/dd770a03-362b-4433-bbe6-50a73e3c683e-kube-api-access-4jmwb\") pod \"multus-additional-cni-plugins-7t5hd\" (UID: \"dd770a03-362b-4433-bbe6-50a73e3c683e\") " pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:00.694195 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.694158 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hhq\" (UniqueName: \"kubernetes.io/projected/e55c9f37-cba5-4014-a612-33d35b24fa8d-kube-api-access-t2hhq\") pod \"tuned-mcsjw\" (UID: \"e55c9f37-cba5-4014-a612-33d35b24fa8d\") " pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.694312 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.694255 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqsfk\" (UniqueName: \"kubernetes.io/projected/ec6d6b7e-bd9c-43ae-a251-e9f911f180d6-kube-api-access-gqsfk\") pod \"ovnkube-node-rlvhw\" (UID: \"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6\") " pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.773109 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.773074 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vk4sh" Jun 08 22:48:00.782232 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.782210 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lcf6r" Jun 08 22:48:00.791270 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.791244 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:00.795205 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.795182 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" Jun 08 22:48:00.803832 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.803809 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-hb47l" Jun 08 22:48:00.809992 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.809972 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-bj7xh" Jun 08 22:48:00.816583 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.816564 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" Jun 08 22:48:00.823138 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.823109 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:00.832587 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:00.832567 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" Jun 08 22:48:01.189755 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.189669 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:01.189755 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.189731 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:01.189983 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.189819 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:01.189983 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.189897 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.189878211 +0000 UTC m=+4.069785289 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:01.189983 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.189821 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:01.189983 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.189964 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.189948718 +0000 UTC m=+4.069855806 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:01.290871 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.290830 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:01.291045 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.291003 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:01.291045 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.291028 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:01.291045 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.291040 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:01.291190 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:01.291098 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:02.291077591 +0000 UTC m=+4.170984660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:01.425081 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.425050 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233ee4f5_e859_436a_966f_db64a39f02ba.slice/crio-eafb723c112db3678524c7c550068fe120e005c63e14975fa43cd7b6125b490d WatchSource:0}: Error finding container eafb723c112db3678524c7c550068fe120e005c63e14975fa43cd7b6125b490d: Status 404 returned error can't find the container with id eafb723c112db3678524c7c550068fe120e005c63e14975fa43cd7b6125b490d Jun 08 22:48:01.425900 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.425872 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7948a206_759b_4031_a1bc_af2b2e94ec55.slice/crio-cdbf5557dc6f2349f81107cd05fbc0ba9d678d0339c0d8ce8b30cf1663a4c7b9 WatchSource:0}: Error finding container cdbf5557dc6f2349f81107cd05fbc0ba9d678d0339c0d8ce8b30cf1663a4c7b9: Status 404 returned error can't find the container with id cdbf5557dc6f2349f81107cd05fbc0ba9d678d0339c0d8ce8b30cf1663a4c7b9 Jun 08 22:48:01.427061 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.427039 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode55c9f37_cba5_4014_a612_33d35b24fa8d.slice/crio-ada1e0b564bfca9b9a6427ba7304cba6e77a21ce5908e8e062626cbd362fb803 WatchSource:0}: Error finding container ada1e0b564bfca9b9a6427ba7304cba6e77a21ce5908e8e062626cbd362fb803: Status 404 returned error can't find the container with id ada1e0b564bfca9b9a6427ba7304cba6e77a21ce5908e8e062626cbd362fb803 Jun 08 22:48:01.430931 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.430907 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6d6b7e_bd9c_43ae_a251_e9f911f180d6.slice/crio-22e143d71eea83bec9a7dc4de5e0e874678137eac921d6cef5032d0061eea44f WatchSource:0}: Error finding container 22e143d71eea83bec9a7dc4de5e0e874678137eac921d6cef5032d0061eea44f: Status 404 returned error can't find the container with id 22e143d71eea83bec9a7dc4de5e0e874678137eac921d6cef5032d0061eea44f Jun 08 22:48:01.433522 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.433495 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c74d00f_6de3_434f_8975_abd5eeda6c8e.slice/crio-79538f839b1fe2aa0ecab3c21282576b7d87c6680d5eade5078942b52ce9c94a WatchSource:0}: Error finding container 79538f839b1fe2aa0ecab3c21282576b7d87c6680d5eade5078942b52ce9c94a: Status 404 returned error can't find the container with id 79538f839b1fe2aa0ecab3c21282576b7d87c6680d5eade5078942b52ce9c94a Jun 08 22:48:01.434284 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.434264 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f666e24_022d_4c5b_b09d_be5ecd0b4c77.slice/crio-57cdd212b92a2320444b83696eb93bbf7f9d4f379d49bdf29817f8718f016d68 WatchSource:0}: Error finding container 57cdd212b92a2320444b83696eb93bbf7f9d4f379d49bdf29817f8718f016d68: Status 404 returned error can't find the container with id 57cdd212b92a2320444b83696eb93bbf7f9d4f379d49bdf29817f8718f016d68 Jun 08 22:48:01.436177 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.436152 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b52ada8_b170_4037_a664_fdd46e23204c.slice/crio-c31e1479a9b29ae17d5a1f0bb081f5d335974f21abc063df30f03df6001d9d86 WatchSource:0}: Error finding container c31e1479a9b29ae17d5a1f0bb081f5d335974f21abc063df30f03df6001d9d86: Status 404 returned error can't find the container with id c31e1479a9b29ae17d5a1f0bb081f5d335974f21abc063df30f03df6001d9d86 Jun 08 22:48:01.436692 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.436662 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd770a03_362b_4433_bbe6_50a73e3c683e.slice/crio-d478d8a876a791e725189e87a0e44adcd180249454aeef211d95109291967600 WatchSource:0}: Error finding container d478d8a876a791e725189e87a0e44adcd180249454aeef211d95109291967600: Status 404 returned error can't find the container with id d478d8a876a791e725189e87a0e44adcd180249454aeef211d95109291967600 Jun 08 22:48:01.437556 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:01.437413 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70d3927_89dd_457d_a77c_2d4c4bdaf48a.slice/crio-68498fc9e0ff457f3b2dc43085a1d8182ffd6ae7ca5f36e4d19a6268c00ccad4 WatchSource:0}: Error finding container 68498fc9e0ff457f3b2dc43085a1d8182ffd6ae7ca5f36e4d19a6268c00ccad4: Status 404 returned error can't find the container with id 68498fc9e0ff457f3b2dc43085a1d8182ffd6ae7ca5f36e4d19a6268c00ccad4 Jun 08 22:48:01.548993 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.548948 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" event={"ID":"b7d005ddae688368c7345a44445a431e","Type":"ContainerStarted","Data":"75f1f997d5eb95ee64b48f6a6218affa198038427a33e56d7a50307fdfd2014b"} Jun 08 22:48:01.551607 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.551578 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerStarted","Data":"d478d8a876a791e725189e87a0e44adcd180249454aeef211d95109291967600"} Jun 08 22:48:01.554489 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.554463 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-htb6v" event={"ID":"4b52ada8-b170-4037-a664-fdd46e23204c","Type":"ContainerStarted","Data":"c31e1479a9b29ae17d5a1f0bb081f5d335974f21abc063df30f03df6001d9d86"} Jun 08 22:48:01.555484 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.555465 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bj7xh" event={"ID":"5c74d00f-6de3-434f-8975-abd5eeda6c8e","Type":"ContainerStarted","Data":"79538f839b1fe2aa0ecab3c21282576b7d87c6680d5eade5078942b52ce9c94a"} Jun 08 22:48:01.556431 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.556410 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" event={"ID":"e55c9f37-cba5-4014-a612-33d35b24fa8d","Type":"ContainerStarted","Data":"ada1e0b564bfca9b9a6427ba7304cba6e77a21ce5908e8e062626cbd362fb803"} Jun 08 22:48:01.557425 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.557404 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vk4sh" event={"ID":"d70d3927-89dd-457d-a77c-2d4c4bdaf48a","Type":"ContainerStarted","Data":"68498fc9e0ff457f3b2dc43085a1d8182ffd6ae7ca5f36e4d19a6268c00ccad4"} Jun 08 22:48:01.561537 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.561491 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-133.ec2.internal" podStartSLOduration=1.56147751 podStartE2EDuration="1.56147751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:01.560887312 +0000 UTC m=+3.440794403" watchObservedRunningTime="2026-06-08 22:48:01.56147751 +0000 UTC m=+3.441384601" Jun 08 22:48:01.563400 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.563379 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" event={"ID":"6f666e24-022d-4c5b-b09d-be5ecd0b4c77","Type":"ContainerStarted","Data":"57cdd212b92a2320444b83696eb93bbf7f9d4f379d49bdf29817f8718f016d68"} Jun 08 22:48:01.567175 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.567134 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"22e143d71eea83bec9a7dc4de5e0e874678137eac921d6cef5032d0061eea44f"} Jun 08 22:48:01.571471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.571431 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lcf6r" event={"ID":"7948a206-759b-4031-a1bc-af2b2e94ec55","Type":"ContainerStarted","Data":"cdbf5557dc6f2349f81107cd05fbc0ba9d678d0339c0d8ce8b30cf1663a4c7b9"} Jun 08 22:48:01.572494 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.572474 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hb47l" event={"ID":"233ee4f5-e859-436a-966f-db64a39f02ba","Type":"ContainerStarted","Data":"eafb723c112db3678524c7c550068fe120e005c63e14975fa43cd7b6125b490d"} Jun 08 22:48:01.810397 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.810351 2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4mp7" Jun 08 22:48:01.820850 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:01.820389 2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4mp7" Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.198296 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.198351 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.198475 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.198523 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.198510699 +0000 UTC m=+6.078417769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.198906 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:02.199105 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.198944 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.198934285 +0000 UTC m=+6.078841355 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:02.299282 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.299230 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:02.311754 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.311719 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:02.311754 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.311761 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:02.311999 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.311783 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:02.311999 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.311907 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:04.311868851 +0000 UTC m=+6.191775938 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:02.540247 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.540171 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:02.540406 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.540321 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:02.540762 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.540744 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:02.540866 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.540846 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:02.541006 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.540992 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:02.541092 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:02.541074 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:02.586867 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.586824 2583 generic.go:356] "Generic (PLEG): container finished" podID="41cd31e5c82cbe87a53c1c85e967347a" containerID="7f0d21adc68156491dbb89bfbe6bc8301668aadda3cca893e1b3c26403f8c6f0" exitCode=0 Jun 08 22:48:02.587936 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.587908 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"41cd31e5c82cbe87a53c1c85e967347a","Type":"ContainerDied","Data":"7f0d21adc68156491dbb89bfbe6bc8301668aadda3cca893e1b3c26403f8c6f0"} Jun 08 22:48:02.824169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.824057 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 22:43:01 +0000 UTC" deadline="2028-03-07 16:38:20.811245024 +0000 UTC" Jun 08 22:48:02.824169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:02.824099 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15305h50m17.987149896s" Jun 08 22:48:03.600527 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:03.599767 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" event={"ID":"41cd31e5c82cbe87a53c1c85e967347a","Type":"ContainerStarted","Data":"133badb112e00405bc955365ebd07a735152b1666388ea096a5b1d289a30fd08"} Jun 08 22:48:03.824355 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:03.824292 2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-07 22:43:01 +0000 UTC" deadline="2027-11-05 23:22:26.342345474 +0000 UTC" Jun 08 22:48:03.824355 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:03.824330 2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12360h34m22.518019385s" Jun 08 22:48:04.216992 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.216949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:04.217191 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.217039 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:04.217258 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.217187 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:04.217258 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.217254 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.217233946 +0000 UTC m=+10.097141017 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:04.217690 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.217670 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:04.217772 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.217726 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.217710084 +0000 UTC m=+10.097617159 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:04.318211 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.318107 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:04.318384 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.318286 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:04.318384 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.318314 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:04.318384 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.318328 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:04.318570 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.318390 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:08.318370416 +0000 UTC m=+10.198277509 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:04.542676 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.542590 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:04.542855 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.542732 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:04.543098 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.543075 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:04.543194 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.543169 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:04.543257 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:04.543216 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:04.543312 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:04.543294 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:06.542238 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:06.542363 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:06.542462 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:06.542546 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:06.542596 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:06.543057 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:06.542664 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:08.255022 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.254979 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:08.255507 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.255069 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:08.255507 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.255215 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:08.255507 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.255277 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.255258236 +0000 UTC m=+18.135165323 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:08.255694 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.255622 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:08.255750 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.255731 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.255664569 +0000 UTC m=+18.135571678 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:08.356331 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.355708 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:08.356331 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.355891 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:08.356331 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.355912 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:08.356331 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.355925 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:08.356331 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.355983 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:16.355964499 +0000 UTC m=+18.235871583 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:08.540237 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.540159 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:08.540382 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.540287 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:08.540835 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.540673 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:08.540835 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.540793 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:08.540835 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:08.540808 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:08.541068 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:08.540966 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:10.543280 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:10.543248 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:10.543280 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:10.543279 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:10.543779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:10.543361 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:10.543779 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:10.543380 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:10.543779 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:10.543502 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:10.543779 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:10.543597 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:12.540171 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:12.540071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:12.540576 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:12.540071 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:12.540576 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:12.540203 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:12.540576 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:12.540308 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:12.540576 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:12.540072 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:12.540576 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:12.540475 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:14.539794 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:14.539755 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:14.540264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:14.539756 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:14.540264 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:14.539905 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:14.540264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:14.539917 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:14.540264 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:14.540012 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:14.540264 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:14.540091 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:16.317941 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.317894 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:16.318462 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.317981 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:16.318462 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.318103 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:16.318462 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.318168 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.318149726 +0000 UTC m=+34.198056810 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:16.318653 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.318568 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:16.318653 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.318607 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.318595517 +0000 UTC m=+34.198502587 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:16.418631 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.418590 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:16.418817 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.418790 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:16.418817 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.418816 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:16.418916 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.418830 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:16.418916 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.418900 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:48:32.418880363 +0000 UTC m=+34.298787442 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:16.540096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.540064 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:16.540285 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.540066 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:16.540285 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.540215 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:16.540285 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:16.540064 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:16.540434 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.540287 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:16.540434 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:16.540359 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:18.540914 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:18.540875 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:18.541339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:18.540960 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:18.541339 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:18.540974 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:18.541339 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:18.541063 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:18.541339 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:18.541084 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:18.541339 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:18.541153 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:20.541633 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:20.541597 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:20.541633 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:20.541633 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:20.542048 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:20.541724 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:20.542048 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:20.541730 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:20.542048 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:20.541792 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:20.542048 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:20.541843 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:21.641282 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.640903 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="f723f420ecdbff4830c81861cf114a333fc0d1bba59f3942f1d2ab7a6a2217de" exitCode=0 Jun 08 22:48:21.642370 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.640994 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"f723f420ecdbff4830c81861cf114a333fc0d1bba59f3942f1d2ab7a6a2217de"} Jun 08 22:48:21.644194 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.644156 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-htb6v" event={"ID":"4b52ada8-b170-4037-a664-fdd46e23204c","Type":"ContainerStarted","Data":"bd7e081bea8aefc92974638de31c281cb20615d0ed733808a90b23bafb5683f2"} Jun 08 22:48:21.647733 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.647700 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-bj7xh" event={"ID":"5c74d00f-6de3-434f-8975-abd5eeda6c8e","Type":"ContainerStarted","Data":"3539cb1798d4844f51d7d99714a824e8986fca968a8763906c562125410f3927"} Jun 08 22:48:21.649501 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.649472 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" event={"ID":"e55c9f37-cba5-4014-a612-33d35b24fa8d","Type":"ContainerStarted","Data":"db32046cc26e11a239cc5f2ef6f2287538de6e6bfae2381088f474aa65487277"} Jun 08 22:48:21.651178 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.651153 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vk4sh" event={"ID":"d70d3927-89dd-457d-a77c-2d4c4bdaf48a","Type":"ContainerStarted","Data":"b728155d608c744d36335d282ed998303f50e7f661e18147c2aafa7dae193a3c"} Jun 08 22:48:21.652754 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.652729 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" event={"ID":"6f666e24-022d-4c5b-b09d-be5ecd0b4c77","Type":"ContainerStarted","Data":"43f168740da40aba0ccada28ba20f0a97b322a5a1e29947704c0cc8de7f5d7b6"} Jun 08 22:48:21.656197 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.656173 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:48:21.657202 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657156 2583 generic.go:356] "Generic (PLEG): container finished" podID="ec6d6b7e-bd9c-43ae-a251-e9f911f180d6" containerID="fb8d336bb23eef6d3f8ef4fa30f6e5cad830f8248cfd48d4ac4b2c83904f9cf4" exitCode=1 Jun 08 22:48:21.657315 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657223 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"5f97a9af4e249cec98f05a9ba4487c23a42d55c4fe14b99476be9226bc464a49"} Jun 08 22:48:21.657315 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657309 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"7b9ba88a36673ecaf2e94f56983fea8829da4b1feb3b0294c621474f6ec80314"} Jun 08 22:48:21.657472 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"ce3a925abaf9ecb8a18f2c28463af7aef01057d5b00616b0edfd1ac5cabdf968"} Jun 08 22:48:21.657472 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657409 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"e0ad3ee95238f25a47cdab142d8ae0f3e2d220646e77d3008834d557f317d0ac"} Jun 08 22:48:21.657597 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657485 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerDied","Data":"fb8d336bb23eef6d3f8ef4fa30f6e5cad830f8248cfd48d4ac4b2c83904f9cf4"} Jun 08 22:48:21.657597 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.657537 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"aac4b5119dfb7154c83695ce913f49b6b821d4cc80cb9251b4fb2858ba50d26b"} Jun 08 22:48:21.658913 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.658880 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lcf6r" event={"ID":"7948a206-759b-4031-a1bc-af2b2e94ec55","Type":"ContainerStarted","Data":"d247956d37cdb2cd55ae944268861a875011b1b0638ffe5e9e5c20defe971222"} Jun 08 22:48:21.663852 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.663805 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-133.ec2.internal" podStartSLOduration=21.663787236 podStartE2EDuration="21.663787236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:48:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:03.616042186 +0000 UTC m=+5.495949279" watchObservedRunningTime="2026-06-08 22:48:21.663787236 +0000 UTC m=+23.543694329" Jun 08 22:48:21.692784 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.692735 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-bj7xh" podStartSLOduration=4.517241942 podStartE2EDuration="23.692719581s" totalImagesPullingTime="19.175477639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.692361689 +0000 UTC m=+23.572268782" watchObservedRunningTime="2026-06-08 22:48:21.692719581 +0000 UTC m=+23.572626673" Jun 08 22:48:21.693017 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.692999 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lcf6r" podStartSLOduration=4.53790096 podStartE2EDuration="23.692992425s" totalImagesPullingTime="19.155091465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.675715878 +0000 UTC m=+23.555622969" watchObservedRunningTime="2026-06-08 22:48:21.692992425 +0000 UTC m=+23.572899517" Jun 08 22:48:21.708524 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.708458 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mcsjw" podStartSLOduration=4.550523934 podStartE2EDuration="23.708422031s" totalImagesPullingTime="19.157898097s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.707963581 +0000 UTC m=+23.587870672" watchObservedRunningTime="2026-06-08 22:48:21.708422031 +0000 UTC m=+23.588329123" Jun 08 22:48:21.722793 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.722736 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-htb6v" podStartSLOduration=12.991241218 podStartE2EDuration="23.722719217s" totalImagesPullingTime="10.731477999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.722071956 +0000 UTC m=+23.601979048" watchObservedRunningTime="2026-06-08 22:48:21.722719217 +0000 UTC m=+23.602626309" Jun 08 22:48:21.735237 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:21.735191 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vk4sh" podStartSLOduration=4.605856195 podStartE2EDuration="23.735177759s" totalImagesPullingTime="19.129321564s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:21.734972223 +0000 UTC m=+23.614879316" watchObservedRunningTime="2026-06-08 22:48:21.735177759 +0000 UTC m=+23.615084851" Jun 08 22:48:22.235392 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.235193 2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 08 22:48:22.539562 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.539466 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:22.539735 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:22.539597 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:22.540329 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.540034 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:22.540329 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:22.540147 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:22.540329 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.540217 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:22.540329 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:22.540296 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:22.555825 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.555711 2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-08T22:48:22.235371457Z","UUID":"27303c8e-6267-4d98-86cc-e8783135d7ca","Handler":null,"Name":"","Endpoint":""} Jun 08 22:48:22.557985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.557521 2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 08 22:48:22.557985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.557565 2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 08 22:48:22.664108 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.664071 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" event={"ID":"6f666e24-022d-4c5b-b09d-be5ecd0b4c77","Type":"ContainerStarted","Data":"1fc2e59ae793003d87e38e83030f29672b0bb789f7abbcdefcac992e1bb43612"} Jun 08 22:48:22.666631 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.666600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-hb47l" event={"ID":"233ee4f5-e859-436a-966f-db64a39f02ba","Type":"ContainerStarted","Data":"5edd5c9d98f53ff3885b59f1c5a2b2ef80f72067442ab2191ec410cceb36075e"} Jun 08 22:48:22.681857 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:22.681797 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-hb47l" podStartSLOduration=5.525220044 podStartE2EDuration="24.681774498s" totalImagesPullingTime="19.156554454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:22.681317021 +0000 UTC m=+24.561224114" watchObservedRunningTime="2026-06-08 22:48:22.681774498 +0000 UTC m=+24.561681598" Jun 08 22:48:23.657287 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.657199 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:23.657927 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.657901 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:23.671200 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.671164 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" event={"ID":"6f666e24-022d-4c5b-b09d-be5ecd0b4c77","Type":"ContainerStarted","Data":"045ce1163155b80d382265293a42d73dd1e6bf3fbeb110a7ffd325f5c3434ea6"} Jun 08 22:48:23.674752 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.674725 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:48:23.675175 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.675147 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"12eaa8c479cb93f907bc434894ab6652f97293e0668f7527c6fad375a9b82a30"} Jun 08 22:48:23.675631 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.675602 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:23.675979 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.675962 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-htb6v" Jun 08 22:48:23.688888 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:23.688835 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xsqzx" podStartSLOduration=4.621193217 podStartE2EDuration="25.688821055s" totalImagesPullingTime="21.067627838s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:23.688304969 +0000 UTC m=+25.568212062" watchObservedRunningTime="2026-06-08 22:48:23.688821055 +0000 UTC m=+25.568728212" Jun 08 22:48:24.540043 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:24.540004 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:24.540254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:24.540064 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:24.540254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:24.540013 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:24.540254 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:24.540153 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:24.540254 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:24.540240 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:24.540495 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:24.540332 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:25.685469 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:25.683086 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:48:26.539332 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.539293 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:26.539332 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.539309 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:26.539559 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:26.539385 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:26.539559 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.539499 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:26.539652 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:26.539587 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:26.539699 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:26.539677 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:26.692964 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.692936 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:48:26.693466 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.693348 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"1559fea9536a8ed41fb3e41ef0afb62d43b6d23f0dfecacd1941b872a92f5c78"} Jun 08 22:48:26.693730 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.693708 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:26.693874 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.693824 2583 scope.go:117] "RemoveContainer" containerID="fb8d336bb23eef6d3f8ef4fa30f6e5cad830f8248cfd48d4ac4b2c83904f9cf4" Jun 08 22:48:26.695632 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.695605 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="ca456055dbfc15fb5fd00b2d6d3d6a3d49c1e6ca22bcc5d8365650c1235e664f" exitCode=0 Jun 08 22:48:26.695726 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.695642 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"ca456055dbfc15fb5fd00b2d6d3d6a3d49c1e6ca22bcc5d8365650c1235e664f"} Jun 08 22:48:26.711839 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:26.711818 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:27.700264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.700082 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="f337b4120118e6aebd68a11e21be218719f772bf4f45461b69d2eccdbcb8ddfb" exitCode=0 Jun 08 22:48:27.700771 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.700162 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"f337b4120118e6aebd68a11e21be218719f772bf4f45461b69d2eccdbcb8ddfb"} Jun 08 22:48:27.704207 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.704189 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:48:27.704587 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.704562 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" event={"ID":"ec6d6b7e-bd9c-43ae-a251-e9f911f180d6","Type":"ContainerStarted","Data":"40a6f72ad586e3d94e33a085995c908bbe39fd969f6967a288b7e675a1e9874a"} Jun 08 22:48:27.704844 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.704824 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:27.704948 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.704904 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:27.721053 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.721027 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:48:27.743121 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.743062 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" podStartSLOduration=10.523619575 podStartE2EDuration="29.743046423s" totalImagesPullingTime="19.219426848s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:27.742986256 +0000 UTC m=+29.622893363" watchObservedRunningTime="2026-06-08 22:48:27.743046423 +0000 UTC m=+29.622953515" Jun 08 22:48:27.862324 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.862269 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fjz6d"] Jun 08 22:48:27.862485 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.862405 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:27.862524 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:27.862507 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:27.862891 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.862873 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lw8qj"] Jun 08 22:48:27.863014 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.862970 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:27.863070 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:27.863053 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:27.873037 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.873011 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7rkj"] Jun 08 22:48:27.873185 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:27.873114 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:27.873239 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:27.873204 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:28.708548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:28.708509 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="66cfe4ccc66f540d89e78ae1cb25811ae84b94ee4ad36878f4c7be567f2ff33b" exitCode=0 Jun 08 22:48:28.709047 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:28.708600 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"66cfe4ccc66f540d89e78ae1cb25811ae84b94ee4ad36878f4c7be567f2ff33b"} Jun 08 22:48:29.539382 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:29.539343 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:29.539592 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:29.539393 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:29.539592 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:29.539394 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:29.539592 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:29.539496 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:29.539743 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:29.539615 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:29.539743 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:29.539693 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:31.539845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:31.539805 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:31.540477 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:31.539940 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-x7rkj" podUID="e1892e47-8908-4673-b782-e55723903d13" Jun 08 22:48:31.540680 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:31.540654 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:31.540811 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:31.540755 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-fjz6d" podUID="9f9782cd-8042-4796-9679-fa21f61de1a0" Jun 08 22:48:31.540880 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:31.540820 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:31.540936 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:31.540903 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:48:32.336511 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:32.336471 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:32.336812 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:32.336553 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:32.336812 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.336614 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:32.336812 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.336646 2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:32.336812 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.336681 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.336662555 +0000 UTC m=+66.216569646 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 08 22:48:32.336812 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.336700 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret podName:e1892e47-8908-4673-b782-e55723903d13 nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.336690633 +0000 UTC m=+66.216597705 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret") pod "global-pull-secret-syncer-x7rkj" (UID: "e1892e47-8908-4673-b782-e55723903d13") : object "kube-system"/"original-pull-secret" not registered Jun 08 22:48:32.437944 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:32.437904 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:32.438142 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.438066 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 08 22:48:32.438142 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.438088 2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 08 22:48:32.438142 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.438100 2583 projected.go:196] Error preparing data for projected volume kube-api-access-rgzmg for pod openshift-network-diagnostics/network-check-target-fjz6d: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:32.438303 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:32.438167 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg podName:9f9782cd-8042-4796-9679-fa21f61de1a0 nodeName:}" failed. No retries permitted until 2026-06-08 22:49:04.438147625 +0000 UTC m=+66.318054717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-rgzmg" (UniqueName: "kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg") pod "network-check-target-fjz6d" (UID: "9f9782cd-8042-4796-9679-fa21f61de1a0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 08 22:48:33.454587 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.454392 2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-133.ec2.internal" event="NodeReady" Jun 08 22:48:33.454955 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.454691 2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 08 22:48:33.493901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.493868 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vvvh2"] Jun 08 22:48:33.506924 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.506885 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-94b4v"] Jun 08 22:48:33.507073 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.507019 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.509668 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.509597 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 08 22:48:33.509668 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.509603 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 08 22:48:33.509856 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.509615 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnlbh\"" Jun 08 22:48:33.531276 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.531243 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvvh2"] Jun 08 22:48:33.531276 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.531283 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-94b4v"] Jun 08 22:48:33.531491 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.531368 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.534002 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.533978 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 08 22:48:33.534002 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.533998 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 08 22:48:33.534221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.534047 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9wnvx\"" Jun 08 22:48:33.534221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.534060 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 08 22:48:33.539374 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.539350 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:48:33.539374 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.539366 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:48:33.539541 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.539362 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:48:33.541948 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.541922 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 22:48:33.542070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.541984 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 22:48:33.542070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.541985 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k2dq5\"" Jun 08 22:48:33.542324 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.542304 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 22:48:33.542854 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.542727 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 22:48:33.542854 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.542822 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjrnx\"" Jun 08 22:48:33.645828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645732 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411ad465-05c7-4a35-8151-b92c1ac48a1e-config-volume\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.645828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645770 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411ad465-05c7-4a35-8151-b92c1ac48a1e-tmp-dir\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.645828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645807 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrn9l\" (UniqueName: \"kubernetes.io/projected/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-kube-api-access-wrn9l\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.645828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645829 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-cert\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.646073 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645844 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpwqq\" (UniqueName: \"kubernetes.io/projected/411ad465-05c7-4a35-8151-b92c1ac48a1e-kube-api-access-xpwqq\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.646073 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.645865 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.723588 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.723551 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="edfbbf08cb52b5dfc264fc306bfb91ee89a1a5d84716975d4ed1cbe8f3249fd3" exitCode=0 Jun 08 22:48:33.723769 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.723607 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"edfbbf08cb52b5dfc264fc306bfb91ee89a1a5d84716975d4ed1cbe8f3249fd3"} Jun 08 22:48:33.746886 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746848 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411ad465-05c7-4a35-8151-b92c1ac48a1e-config-volume\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.746886 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746882 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411ad465-05c7-4a35-8151-b92c1ac48a1e-tmp-dir\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.747126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrn9l\" (UniqueName: \"kubernetes.io/projected/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-kube-api-access-wrn9l\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.747126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746933 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-cert\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.747126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746948 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpwqq\" (UniqueName: \"kubernetes.io/projected/411ad465-05c7-4a35-8151-b92c1ac48a1e-kube-api-access-xpwqq\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.747126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.746968 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.747126 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:33.747097 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:33.747354 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:33.747159 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:34.247140705 +0000 UTC m=+36.127047795 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:33.747354 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.747282 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411ad465-05c7-4a35-8151-b92c1ac48a1e-tmp-dir\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.747573 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.747549 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/411ad465-05c7-4a35-8151-b92c1ac48a1e-config-volume\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.751507 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.751466 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-cert\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.755414 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.755392 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpwqq\" (UniqueName: \"kubernetes.io/projected/411ad465-05c7-4a35-8151-b92c1ac48a1e-kube-api-access-xpwqq\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:33.755614 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.755595 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrn9l\" (UniqueName: \"kubernetes.io/projected/f91f94b7-610c-4949-8a36-e3cf1f35c9d6-kube-api-access-wrn9l\") pod \"ingress-canary-94b4v\" (UID: \"f91f94b7-610c-4949-8a36-e3cf1f35c9d6\") " pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:33.840643 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:33.840599 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-94b4v" Jun 08 22:48:34.013593 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:34.013564 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-94b4v"] Jun 08 22:48:34.017138 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:34.017112 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91f94b7_610c_4949_8a36_e3cf1f35c9d6.slice/crio-117b4636b687373cd89f4419ac684d38d3cffff756ffb906e6f91f429bd067a1 WatchSource:0}: Error finding container 117b4636b687373cd89f4419ac684d38d3cffff756ffb906e6f91f429bd067a1: Status 404 returned error can't find the container with id 117b4636b687373cd89f4419ac684d38d3cffff756ffb906e6f91f429bd067a1 Jun 08 22:48:34.250985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:34.250892 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:34.251136 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:34.251044 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:34.251136 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:34.251112 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:35.251096785 +0000 UTC m=+37.131003856 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:34.727374 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:34.727337 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-94b4v" event={"ID":"f91f94b7-610c-4949-8a36-e3cf1f35c9d6","Type":"ContainerStarted","Data":"117b4636b687373cd89f4419ac684d38d3cffff756ffb906e6f91f429bd067a1"} Jun 08 22:48:34.731049 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:34.731021 2583 generic.go:356] "Generic (PLEG): container finished" podID="dd770a03-362b-4433-bbe6-50a73e3c683e" containerID="ca9c6fddeb25a2e18def7ab54233f931a816970d2c603d913833eae08792ebe8" exitCode=0 Jun 08 22:48:34.731195 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:34.731078 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerDied","Data":"ca9c6fddeb25a2e18def7ab54233f931a816970d2c603d913833eae08792ebe8"} Jun 08 22:48:35.260286 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:35.260248 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:35.260503 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:35.260434 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:35.260552 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:35.260540 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:37.260517012 +0000 UTC m=+39.140424095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:35.736566 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:35.736531 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" event={"ID":"dd770a03-362b-4433-bbe6-50a73e3c683e","Type":"ContainerStarted","Data":"a2df60a4ed17e772a2b23c2d07c019cd4fd50e9258f324a2d8c1b31c78590b19"} Jun 08 22:48:35.758787 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:35.758734 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7t5hd" podStartSLOduration=9.131506189 podStartE2EDuration="37.758719244s" totalImagesPullingTime="28.627213055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:35.757152595 +0000 UTC m=+37.637059687" watchObservedRunningTime="2026-06-08 22:48:35.758719244 +0000 UTC m=+37.638626336" Jun 08 22:48:36.740459 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:36.740403 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-94b4v" event={"ID":"f91f94b7-610c-4949-8a36-e3cf1f35c9d6","Type":"ContainerStarted","Data":"cc500f013e9593c22c9f6f14f4c614a4e0d6432f8ae508bb32e45adce94ef97b"} Jun 08 22:48:36.756430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:36.756377 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-94b4v" podStartSLOduration=1.843393439 podStartE2EDuration="3.756360454s" totalImagesPullingTime="1.912967015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:48:36.75568233 +0000 UTC m=+38.635589419" watchObservedRunningTime="2026-06-08 22:48:36.756360454 +0000 UTC m=+38.636267572" Jun 08 22:48:37.275848 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:37.275805 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:37.276017 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:37.275967 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:37.276057 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:37.276040 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:41.276022154 +0000 UTC m=+43.155929238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:41.301721 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:41.301678 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:41.302213 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:41.301845 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:41.302213 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:41.301912 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:48:49.301895157 +0000 UTC m=+51.181802228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:49.358000 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:49.357949 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:48:49.358504 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:49.358125 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:48:49.358504 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:48:49.358218 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:49:05.358199231 +0000 UTC m=+67.238106301 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:48:56.269547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.269511 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq"] Jun 08 22:48:56.273126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.273104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.275970 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.275943 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 08 22:48:56.277071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.277052 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 08 22:48:56.277173 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.277069 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 08 22:48:56.277173 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.277054 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 08 22:48:56.281956 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.281932 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq"] Jun 08 22:48:56.406829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.406790 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgt7v\" (UniqueName: \"kubernetes.io/projected/a5d99849-61ca-440f-97e3-c80b1b5d3a84-kube-api-access-xgt7v\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.407021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.406848 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5d99849-61ca-440f-97e3-c80b1b5d3a84-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.407021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.406922 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d99849-61ca-440f-97e3-c80b1b5d3a84-tmp\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.507314 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.507265 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgt7v\" (UniqueName: \"kubernetes.io/projected/a5d99849-61ca-440f-97e3-c80b1b5d3a84-kube-api-access-xgt7v\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.507556 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.507337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5d99849-61ca-440f-97e3-c80b1b5d3a84-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.507556 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.507397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d99849-61ca-440f-97e3-c80b1b5d3a84-tmp\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.507827 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.507804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a5d99849-61ca-440f-97e3-c80b1b5d3a84-tmp\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.511557 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.511531 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a5d99849-61ca-440f-97e3-c80b1b5d3a84-klusterlet-config\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.514935 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.514901 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgt7v\" (UniqueName: \"kubernetes.io/projected/a5d99849-61ca-440f-97e3-c80b1b5d3a84-kube-api-access-xgt7v\") pod \"klusterlet-addon-workmgr-66c46df55b-5nbgq\" (UID: \"a5d99849-61ca-440f-97e3-c80b1b5d3a84\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.583903 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.583858 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:48:56.719035 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.719000 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq"] Jun 08 22:48:56.722517 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:48:56.722483 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d99849_61ca_440f_97e3_c80b1b5d3a84.slice/crio-9a9735d87de94f804f0338450542575ce9753389e760f657861b8daf11b35f09 WatchSource:0}: Error finding container 9a9735d87de94f804f0338450542575ce9753389e760f657861b8daf11b35f09: Status 404 returned error can't find the container with id 9a9735d87de94f804f0338450542575ce9753389e760f657861b8daf11b35f09 Jun 08 22:48:56.786313 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:56.786276 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" event={"ID":"a5d99849-61ca-440f-97e3-c80b1b5d3a84","Type":"ContainerStarted","Data":"9a9735d87de94f804f0338450542575ce9753389e760f657861b8daf11b35f09"} Jun 08 22:48:59.723296 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:48:59.723267 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-rlvhw" Jun 08 22:49:00.800432 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:00.800387 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" event={"ID":"a5d99849-61ca-440f-97e3-c80b1b5d3a84","Type":"ContainerStarted","Data":"8cef6a6706b0de6c6f744695fb2cf5bba207d5c4942a6b48696550e71d2f235b"} Jun 08 22:49:00.800973 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:00.800566 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:49:00.802975 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:00.802947 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:49:00.814918 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:00.814871 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" podStartSLOduration=0.834451157 podStartE2EDuration="4.814853372s" totalImagesPullingTime="3.980402215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:00.813985339 +0000 UTC m=+62.693892444" watchObservedRunningTime="2026-06-08 22:49:00.814853372 +0000 UTC m=+62.694760464" Jun 08 22:49:04.361670 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.361625 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:49:04.362163 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.361705 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:49:04.364074 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.364052 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 08 22:49:04.364160 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.364091 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 08 22:49:04.372232 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:04.372199 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 22:49:04.372361 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:04.372280 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:50:08.372263057 +0000 UTC m=+130.252170128 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : secret "metrics-daemon-secret" not found Jun 08 22:49:04.374719 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.374686 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1892e47-8908-4673-b782-e55723903d13-original-pull-secret\") pod \"global-pull-secret-syncer-x7rkj\" (UID: \"e1892e47-8908-4673-b782-e55723903d13\") " pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:49:04.451967 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.451930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-x7rkj" Jun 08 22:49:04.462427 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.462398 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:49:04.465121 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.465103 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 08 22:49:04.475208 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.475187 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 08 22:49:04.486221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.486200 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rgzmg\" (UniqueName: \"kubernetes.io/projected/9f9782cd-8042-4796-9679-fa21f61de1a0-kube-api-access-rgzmg\") pod \"network-check-target-fjz6d\" (UID: \"9f9782cd-8042-4796-9679-fa21f61de1a0\") " pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:49:04.595751 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.595627 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-x7rkj"] Jun 08 22:49:04.598344 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:49:04.598315 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1892e47_8908_4673_b782_e55723903d13.slice/crio-404d70f7340516279daeba21831faf2cef8579d2065c8dffd57fd982c302de50 WatchSource:0}: Error finding container 404d70f7340516279daeba21831faf2cef8579d2065c8dffd57fd982c302de50: Status 404 returned error can't find the container with id 404d70f7340516279daeba21831faf2cef8579d2065c8dffd57fd982c302de50 Jun 08 22:49:04.760001 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.759920 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k2dq5\"" Jun 08 22:49:04.768147 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.768116 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:49:04.814981 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.814876 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7rkj" event={"ID":"e1892e47-8908-4673-b782-e55723903d13","Type":"ContainerStarted","Data":"404d70f7340516279daeba21831faf2cef8579d2065c8dffd57fd982c302de50"} Jun 08 22:49:04.919423 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:04.919387 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-fjz6d"] Jun 08 22:49:04.923157 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:49:04.923114 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f9782cd_8042_4796_9679_fa21f61de1a0.slice/crio-411fee4b46c6b03241f9a9dfe3873202cd600f1aeb53d49bde2b7dc514b486bd WatchSource:0}: Error finding container 411fee4b46c6b03241f9a9dfe3873202cd600f1aeb53d49bde2b7dc514b486bd: Status 404 returned error can't find the container with id 411fee4b46c6b03241f9a9dfe3873202cd600f1aeb53d49bde2b7dc514b486bd Jun 08 22:49:05.368046 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:05.367994 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:49:05.368555 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:05.368150 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:49:05.368555 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:05.368234 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:49:37.368215218 +0000 UTC m=+99.248122287 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:49:05.820421 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:05.820381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fjz6d" event={"ID":"9f9782cd-8042-4796-9679-fa21f61de1a0","Type":"ContainerStarted","Data":"411fee4b46c6b03241f9a9dfe3873202cd600f1aeb53d49bde2b7dc514b486bd"} Jun 08 22:49:09.832794 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:09.832757 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-fjz6d" event={"ID":"9f9782cd-8042-4796-9679-fa21f61de1a0","Type":"ContainerStarted","Data":"3ad42ee206ecdc61de8ff4aa093b6cf306aab09fd80f2b22248e572baad66a75"} Jun 08 22:49:09.833265 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:09.832834 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:49:09.834297 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:09.834270 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-x7rkj" event={"ID":"e1892e47-8908-4673-b782-e55723903d13","Type":"ContainerStarted","Data":"20abd663142d2ac5b74561bb31bc9451c5c53790274e0a588064b68f9566d99b"} Jun 08 22:49:09.847834 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:09.847777 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-fjz6d" podStartSLOduration=67.552350849 podStartE2EDuration="1m11.847761301s" totalImagesPullingTime="4.295410452s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:09.846738318 +0000 UTC m=+71.726645412" watchObservedRunningTime="2026-06-08 22:49:09.847761301 +0000 UTC m=+71.727668393" Jun 08 22:49:09.862546 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:09.860279 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-x7rkj" podStartSLOduration=67.236043371 podStartE2EDuration="1m11.860259155s" totalImagesPullingTime="4.624215784s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:49:09.860016529 +0000 UTC m=+71.739923619" watchObservedRunningTime="2026-06-08 22:49:09.860259155 +0000 UTC m=+71.740166247" Jun 08 22:49:37.391999 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:37.391943 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:49:37.392395 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:37.392110 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:49:37.392395 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:49:37.392190 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:50:41.392174872 +0000 UTC m=+163.272081942 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:49:40.839240 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:49:40.839207 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-fjz6d" Jun 08 22:50:08.391566 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:50:08.391523 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:50:08.392040 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:08.391664 2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 08 22:50:08.392040 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:08.391741 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs podName:cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c nodeName:}" failed. No retries permitted until 2026-06-08 22:52:10.391725584 +0000 UTC m=+252.271632655 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs") pod "network-metrics-daemon-lw8qj" (UID: "cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c") : secret "metrics-daemon-secret" not found Jun 08 22:50:36.518247 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:36.518145 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vvvh2" podUID="411ad465-05c7-4a35-8151-b92c1ac48a1e" Jun 08 22:50:36.562627 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:36.562586 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-lw8qj" podUID="cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c" Jun 08 22:50:37.049695 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:50:37.049664 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:50:41.424387 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:50:41.424339 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:50:41.424798 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:41.424537 2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 08 22:50:41.424798 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:50:41.424618 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls podName:411ad465-05c7-4a35-8151-b92c1ac48a1e nodeName:}" failed. No retries permitted until 2026-06-08 22:52:43.424602126 +0000 UTC m=+285.304509200 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls") pod "dns-default-vvvh2" (UID: "411ad465-05c7-4a35-8151-b92c1ac48a1e") : secret "dns-default-metrics-tls" not found Jun 08 22:50:47.539630 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:50:47.539590 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:51:01.107791 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:01.107699 2583 generic.go:356] "Generic (PLEG): container finished" podID="a5d99849-61ca-440f-97e3-c80b1b5d3a84" containerID="8cef6a6706b0de6c6f744695fb2cf5bba207d5c4942a6b48696550e71d2f235b" exitCode=1 Jun 08 22:51:01.107791 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:01.107744 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" event={"ID":"a5d99849-61ca-440f-97e3-c80b1b5d3a84","Type":"ContainerDied","Data":"8cef6a6706b0de6c6f744695fb2cf5bba207d5c4942a6b48696550e71d2f235b"} Jun 08 22:51:01.108231 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:01.108034 2583 scope.go:117] "RemoveContainer" containerID="8cef6a6706b0de6c6f744695fb2cf5bba207d5c4942a6b48696550e71d2f235b" Jun 08 22:51:02.116922 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:02.116886 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" event={"ID":"a5d99849-61ca-440f-97e3-c80b1b5d3a84","Type":"ContainerStarted","Data":"eef1ea46187aeca5039ff38a3b247618d11f7383d2db3da158e570e41a5fcd52"} Jun 08 22:51:02.117318 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:02.117198 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:51:02.118422 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:02.118008 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-66c46df55b-5nbgq" Jun 08 22:51:11.753586 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.753549 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-79cdcd997f-qf4wj"] Jun 08 22:51:11.756175 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.756156 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:11.759040 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759017 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 08 22:51:11.759175 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759021 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 08 22:51:11.759365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759341 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 08 22:51:11.759424 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759364 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 08 22:51:11.759573 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759553 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 08 22:51:11.759645 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759584 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-ff748\"" Jun 08 22:51:11.759694 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.759642 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 08 22:51:11.774962 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.774932 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79cdcd997f-qf4wj"] Jun 08 22:51:11.847065 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.847025 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8"] Jun 08 22:51:11.850016 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.849988 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" Jun 08 22:51:11.851385 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.851359 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-896549976-hmw8f"] Jun 08 22:51:11.853028 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.853009 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-v8ks8\"" Jun 08 22:51:11.853921 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.853899 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f7f5bcd9f-28z5t"] Jun 08 22:51:11.854027 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.854011 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:11.856637 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.856616 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:11.871711 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.871686 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jun 08 22:51:11.886338 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.886306 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 22:51:11.886517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.886476 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8"] Jun 08 22:51:11.888255 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.888235 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jun 08 22:51:11.889546 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.889532 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 08 22:51:11.892025 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.892009 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpm8k\"" Jun 08 22:51:11.892693 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.892679 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-vt89h\"" Jun 08 22:51:11.893047 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.893035 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 22:51:11.893214 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.893203 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 22:51:11.894779 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.894762 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 08 22:51:11.906763 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.906730 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hmw8f"] Jun 08 22:51:11.907173 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.907154 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jun 08 22:51:11.909435 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.908168 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 22:51:11.914263 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.914226 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f7f5bcd9f-28z5t"] Jun 08 22:51:11.934226 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.934195 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-default-certificate\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:11.934471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.934250 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:11.934471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.934276 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:11.934471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.934306 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j8l8n\" (UniqueName: \"kubernetes.io/projected/5148df69-7982-4755-bbe5-6f131cf83393-kube-api-access-j8l8n\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:11.934471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:11.934326 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-stats-auth\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.035618 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035522 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-stats-auth\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.035618 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035573 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0df19849-731a-4b98-b3a4-26739a64e434-serving-cert\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.035618 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035593 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-service-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.035618 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035612 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-trusted-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.035865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035661 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.035865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035765 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.035865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035804 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.035865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035832 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr9wq\" (UniqueName: \"kubernetes.io/projected/0df19849-731a-4b98-b3a4-26739a64e434-kube-api-access-xr9wq\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.036021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035864 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.036021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035888 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-default-certificate\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.036021 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.035926 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.036165 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036049 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-snapshots\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.036165 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-tmp\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.036165 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036103 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqm6z\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.036165 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036119 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.036165 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036133 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sdcm\" (UniqueName: \"kubernetes.io/projected/ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee-kube-api-access-8sdcm\") pod \"network-check-source-dbd9cf844-kldp8\" (UID: \"ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" Jun 08 22:51:12.036382 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.036382 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036198 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.036382 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036273 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.036382 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.036295 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:12.536273401 +0000 UTC m=+194.416180472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:12.036382 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.036336 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 22:51:12.036633 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.036488 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:12.536471549 +0000 UTC m=+194.416378618 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : secret "router-metrics-certs-default" not found Jun 08 22:51:12.036633 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.036528 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j8l8n\" (UniqueName: \"kubernetes.io/projected/5148df69-7982-4755-bbe5-6f131cf83393-kube-api-access-j8l8n\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.038351 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.038323 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-stats-auth\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.038507 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.038350 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-default-certificate\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.044963 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.044941 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j8l8n\" (UniqueName: \"kubernetes.io/projected/5148df69-7982-4755-bbe5-6f131cf83393-kube-api-access-j8l8n\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.137126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137084 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137128 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-snapshots\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137153 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-tmp\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137172 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqm6z\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137189 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137207 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8sdcm\" (UniqueName: \"kubernetes.io/projected/ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee-kube-api-access-8sdcm\") pod \"network-check-source-dbd9cf844-kldp8\" (UID: \"ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.137311 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.137333 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7f5bcd9f-28z5t: secret "image-registry-tls" not found Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137394 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137403 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.137404 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls podName:552d6379-2430-43c9-af3e-d93c91c16a07 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:12.637385691 +0000 UTC m=+194.517292761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls") pod "image-registry-5f7f5bcd9f-28z5t" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07") : secret "image-registry-tls" not found Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137470 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0df19849-731a-4b98-b3a4-26739a64e434-serving-cert\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137494 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-service-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-trusted-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137536 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137660 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137710 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137753 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr9wq\" (UniqueName: \"kubernetes.io/projected/0df19849-731a-4b98-b3a4-26739a64e434-kube-api-access-xr9wq\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.137840 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137795 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.138243 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.137867 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-tmp\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.138324 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138296 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-service-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.138553 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138486 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.138646 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138594 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.138818 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138787 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0df19849-731a-4b98-b3a4-26739a64e434-trusted-ca-bundle\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.138940 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138790 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/0df19849-731a-4b98-b3a4-26739a64e434-snapshots\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.138940 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.138799 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.140199 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.140171 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0df19849-731a-4b98-b3a4-26739a64e434-serving-cert\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.140318 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.140293 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.140410 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.140387 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.146333 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.146280 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.146649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.146623 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sdcm\" (UniqueName: \"kubernetes.io/projected/ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee-kube-api-access-8sdcm\") pod \"network-check-source-dbd9cf844-kldp8\" (UID: \"ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee\") " pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" Jun 08 22:51:12.146786 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.146629 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqm6z\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.147581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.147548 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr9wq\" (UniqueName: \"kubernetes.io/projected/0df19849-731a-4b98-b3a4-26739a64e434-kube-api-access-xr9wq\") pod \"insights-operator-896549976-hmw8f\" (UID: \"0df19849-731a-4b98-b3a4-26739a64e434\") " pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.160544 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.160508 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" Jun 08 22:51:12.165832 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.165809 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-896549976-hmw8f" Jun 08 22:51:12.294740 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.294713 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8"] Jun 08 22:51:12.297283 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:12.297254 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddfd0afb_c6b2_4566_844c_e4fc8d7a8aee.slice/crio-c2be2e29519ef4f71b924b56502d17dc1e4f6f28806170a7fec1d57595142eef WatchSource:0}: Error finding container c2be2e29519ef4f71b924b56502d17dc1e4f6f28806170a7fec1d57595142eef: Status 404 returned error can't find the container with id c2be2e29519ef4f71b924b56502d17dc1e4f6f28806170a7fec1d57595142eef Jun 08 22:51:12.316987 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.316960 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-896549976-hmw8f"] Jun 08 22:51:12.323483 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:12.323431 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0df19849_731a_4b98_b3a4_26739a64e434.slice/crio-22a8fb2f0068f7e31ce09256101a3a70b0aac9f4626784c4a2590a8fe7d158db WatchSource:0}: Error finding container 22a8fb2f0068f7e31ce09256101a3a70b0aac9f4626784c4a2590a8fe7d158db: Status 404 returned error can't find the container with id 22a8fb2f0068f7e31ce09256101a3a70b0aac9f4626784c4a2590a8fe7d158db Jun 08 22:51:12.541252 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.541213 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.541252 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.541257 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:12.541521 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.541364 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 22:51:12.541521 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.541423 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:13.541400457 +0000 UTC m=+195.421307531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:12.541521 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.541473 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:13.541460731 +0000 UTC m=+195.421367801 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : secret "router-metrics-certs-default" not found Jun 08 22:51:12.642562 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:12.642472 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:12.642706 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.642622 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 22:51:12.642706 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.642638 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7f5bcd9f-28z5t: secret "image-registry-tls" not found Jun 08 22:51:12.642706 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:12.642694 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls podName:552d6379-2430-43c9-af3e-d93c91c16a07 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:13.642677436 +0000 UTC m=+195.522584509 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls") pod "image-registry-5f7f5bcd9f-28z5t" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07") : secret "image-registry-tls" not found Jun 08 22:51:13.152001 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.151958 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hmw8f" event={"ID":"0df19849-731a-4b98-b3a4-26739a64e434","Type":"ContainerStarted","Data":"22a8fb2f0068f7e31ce09256101a3a70b0aac9f4626784c4a2590a8fe7d158db"} Jun 08 22:51:13.153328 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.153307 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" event={"ID":"ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee","Type":"ContainerStarted","Data":"88d4c39658e4fcfdd1c8a722a11a22ac99234a00f7c8356a7afc039e30647e12"} Jun 08 22:51:13.153392 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.153346 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" event={"ID":"ddfd0afb-c6b2-4566-844c-e4fc8d7a8aee","Type":"ContainerStarted","Data":"c2be2e29519ef4f71b924b56502d17dc1e4f6f28806170a7fec1d57595142eef"} Jun 08 22:51:13.168644 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.168591 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-dbd9cf844-kldp8" podStartSLOduration=2.168571417 podStartE2EDuration="2.168571417s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:13.167763603 +0000 UTC m=+195.047670694" watchObservedRunningTime="2026-06-08 22:51:13.168571417 +0000 UTC m=+195.048478510" Jun 08 22:51:13.548854 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.548816 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:13.549052 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.548860 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:13.549052 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.548982 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:15.548963093 +0000 UTC m=+197.428870176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:13.549052 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.549016 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 22:51:13.549212 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.549086 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:15.549066724 +0000 UTC m=+197.428973808 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : secret "router-metrics-certs-default" not found Jun 08 22:51:13.649551 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:13.649506 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:13.649705 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.649653 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 22:51:13.649705 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.649673 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7f5bcd9f-28z5t: secret "image-registry-tls" not found Jun 08 22:51:13.649796 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:13.649728 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls podName:552d6379-2430-43c9-af3e-d93c91c16a07 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:15.64971376 +0000 UTC m=+197.529620831 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls") pod "image-registry-5f7f5bcd9f-28z5t" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07") : secret "image-registry-tls" not found Jun 08 22:51:15.160201 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:15.160107 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hmw8f" event={"ID":"0df19849-731a-4b98-b3a4-26739a64e434","Type":"ContainerStarted","Data":"efb9e5aa697b5b7fbfca0abd8c9c8071f4c83d7d5edba34dc2b095c6c820868a"} Jun 08 22:51:15.176039 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:15.175986 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-896549976-hmw8f" podStartSLOduration=1.627693833 podStartE2EDuration="4.175969467s" totalImagesPullingTime="2.548275634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:15.17484381 +0000 UTC m=+197.054750914" watchObservedRunningTime="2026-06-08 22:51:15.175969467 +0000 UTC m=+197.055876574" Jun 08 22:51:15.567396 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:15.567358 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:15.567396 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:15.567397 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:15.567771 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.567572 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 22:51:15.567771 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.567588 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:19.56756459 +0000 UTC m=+201.447471671 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:15.567771 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.567616 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:19.567605809 +0000 UTC m=+201.447512880 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : secret "router-metrics-certs-default" not found Jun 08 22:51:15.667949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:15.667912 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:15.668101 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.668055 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 22:51:15.668101 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.668071 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7f5bcd9f-28z5t: secret "image-registry-tls" not found Jun 08 22:51:15.668181 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:15.668132 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls podName:552d6379-2430-43c9-af3e-d93c91c16a07 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:19.668113684 +0000 UTC m=+201.548020757 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls") pod "image-registry-5f7f5bcd9f-28z5t" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07") : secret "image-registry-tls" not found Jun 08 22:51:18.559419 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.559379 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-sc62p"] Jun 08 22:51:18.562826 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.562809 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.565377 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.565348 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 08 22:51:18.565377 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.565374 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 08 22:51:18.566504 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.566474 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-6wmw4\"" Jun 08 22:51:18.566630 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.566504 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 08 22:51:18.566630 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.566560 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 08 22:51:18.569628 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.569536 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-sc62p"] Jun 08 22:51:18.590352 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.590315 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-cabundle\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.590352 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.590356 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnj89\" (UniqueName: \"kubernetes.io/projected/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-kube-api-access-hnj89\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.590587 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.590378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-key\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.690886 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.690841 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-cabundle\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.691056 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.690895 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnj89\" (UniqueName: \"kubernetes.io/projected/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-kube-api-access-hnj89\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.691056 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.690929 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-key\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.691658 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.691630 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-cabundle\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.693514 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.693485 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-signing-key\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.698831 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.698804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnj89\" (UniqueName: \"kubernetes.io/projected/d2aee0ec-990e-45e4-9c23-e1b7f422d9d0-kube-api-access-hnj89\") pod \"service-ca-7d74d578b8-sc62p\" (UID: \"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0\") " pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:18.873136 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:18.873042 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" Jun 08 22:51:19.007172 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.007061 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7d74d578b8-sc62p"] Jun 08 22:51:19.009565 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:19.009533 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2aee0ec_990e_45e4_9c23_e1b7f422d9d0.slice/crio-b4510e8757b17f7aca9e2fe4601c109483a425f3ba789be4ab339034ec3fb115 WatchSource:0}: Error finding container b4510e8757b17f7aca9e2fe4601c109483a425f3ba789be4ab339034ec3fb115: Status 404 returned error can't find the container with id b4510e8757b17f7aca9e2fe4601c109483a425f3ba789be4ab339034ec3fb115 Jun 08 22:51:19.175309 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.175225 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" event={"ID":"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0","Type":"ContainerStarted","Data":"b4510e8757b17f7aca9e2fe4601c109483a425f3ba789be4ab339034ec3fb115"} Jun 08 22:51:19.360778 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.360745 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vk4sh_d70d3927-89dd-457d-a77c-2d4c4bdaf48a/dns-node-resolver/0.log" Jun 08 22:51:19.596806 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.596767 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:19.597301 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.596822 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:19.597301 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.596925 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:27.596900104 +0000 UTC m=+209.476807222 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:19.597301 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.596992 2583 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 08 22:51:19.597301 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.597052 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:27.59703389 +0000 UTC m=+209.476940963 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : secret "router-metrics-certs-default" not found Jun 08 22:51:19.697956 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:19.697916 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") pod \"image-registry-5f7f5bcd9f-28z5t\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:19.698147 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.698096 2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 08 22:51:19.698147 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.698119 2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f7f5bcd9f-28z5t: secret "image-registry-tls" not found Jun 08 22:51:19.698275 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:19.698190 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls podName:552d6379-2430-43c9-af3e-d93c91c16a07 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:27.698168611 +0000 UTC m=+209.578075691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls") pod "image-registry-5f7f5bcd9f-28z5t" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07") : secret "image-registry-tls" not found Jun 08 22:51:20.361197 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:20.361168 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lcf6r_7948a206-759b-4031-a1bc-af2b2e94ec55/node-ca/0.log" Jun 08 22:51:20.760490 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:20.760470 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-94b4v_f91f94b7-610c-4949-8a36-e3cf1f35c9d6/serve-healthcheck-canary/0.log" Jun 08 22:51:21.189535 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:21.189493 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" event={"ID":"d2aee0ec-990e-45e4-9c23-e1b7f422d9d0","Type":"ContainerStarted","Data":"8a2afdb3461626ce5a1046cb646c329b7e49bc090edce99bef18719c0b94a6d5"} Jun 08 22:51:25.496161 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:25.496098 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7d74d578b8-sc62p" podStartSLOduration=5.756572621 podStartE2EDuration="7.496080878s" totalImagesPullingTime="1.739508257s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:21.203761575 +0000 UTC m=+203.083668666" watchObservedRunningTime="2026-06-08 22:51:25.496080878 +0000 UTC m=+207.375987969" Jun 08 22:51:25.496837 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:25.496815 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7f5bcd9f-28z5t"] Jun 08 22:51:25.497069 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:25.497049 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" podUID="552d6379-2430-43c9-af3e-d93c91c16a07" Jun 08 22:51:26.204701 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.204663 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:26.211342 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.211315 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:26.252468 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252416 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252492 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252524 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252550 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252604 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252656 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqm6z\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252687 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted\") pod \"552d6379-2430-43c9-af3e-d93c91c16a07\" (UID: \"552d6379-2430-43c9-af3e-d93c91c16a07\") " Jun 08 22:51:26.252966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252861 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:51:26.252966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.252895 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:51:26.253106 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.253075 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 08 22:51:26.255244 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.255212 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z" (OuterVolumeSpecName: "kube-api-access-fqm6z") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "kube-api-access-fqm6z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:51:26.255377 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.255331 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:51:26.255528 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.255481 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:51:26.255647 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.255630 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "552d6379-2430-43c9-af3e-d93c91c16a07" (UID: "552d6379-2430-43c9-af3e-d93c91c16a07"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:51:26.353825 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353787 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqm6z\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-kube-api-access-fqm6z\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.353825 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353820 2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/552d6379-2430-43c9-af3e-d93c91c16a07-ca-trust-extracted\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.353825 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353829 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-trusted-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.354107 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353840 2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/552d6379-2430-43c9-af3e-d93c91c16a07-registry-certificates\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.354107 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353868 2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-image-registry-private-configuration\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.354107 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353881 2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-bound-sa-token\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:26.354107 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:26.353896 2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/552d6379-2430-43c9-af3e-d93c91c16a07-installation-pull-secrets\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:27.207527 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.207493 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f7f5bcd9f-28z5t" Jun 08 22:51:27.236095 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.236062 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f7f5bcd9f-28z5t"] Jun 08 22:51:27.239615 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.239576 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f7f5bcd9f-28z5t"] Jun 08 22:51:27.260620 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.260588 2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/552d6379-2430-43c9-af3e-d93c91c16a07-registry-tls\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:51:27.663926 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.663880 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:27.664350 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.663935 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:27.668189 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:27.664680 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle podName:5148df69-7982-4755-bbe5-6f131cf83393 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:43.664649363 +0000 UTC m=+225.544556437 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle") pod "router-default-79cdcd997f-qf4wj" (UID: "5148df69-7982-4755-bbe5-6f131cf83393") : configmap references non-existent config key: service-ca.crt Jun 08 22:51:27.671102 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:27.671075 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5148df69-7982-4755-bbe5-6f131cf83393-metrics-certs\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:28.118427 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.118392 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c48d6f8c8-vv7p8"] Jun 08 22:51:28.121691 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.121674 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.124309 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.124284 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 08 22:51:28.124474 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.124366 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 08 22:51:28.124474 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.124366 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpm8k\"" Jun 08 22:51:28.124689 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.124670 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 08 22:51:28.129076 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.128691 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 08 22:51:28.135987 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.135956 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c48d6f8c8-vv7p8"] Jun 08 22:51:28.168799 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168766 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-trusted-ca\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168799 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168802 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-registry-tls\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168822 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-registry-certificates\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168841 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-image-registry-private-configuration\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-bound-sa-token\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168933 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-installation-pull-secrets\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.168985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168951 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12d607d3-5fcc-4357-bd03-1c9697641926-ca-trust-extracted\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.169169 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.168999 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6lbp\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-kube-api-access-z6lbp\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270300 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-bound-sa-token\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270300 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270311 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-installation-pull-secrets\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270349 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12d607d3-5fcc-4357-bd03-1c9697641926-ca-trust-extracted\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270378 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6lbp\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-kube-api-access-z6lbp\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270434 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-trusted-ca\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270480 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-registry-tls\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270502 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-registry-certificates\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.270901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.270534 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-image-registry-private-configuration\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.271428 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.271390 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-trusted-ca\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.271536 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.271428 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/12d607d3-5fcc-4357-bd03-1c9697641926-registry-certificates\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.271597 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.271584 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/12d607d3-5fcc-4357-bd03-1c9697641926-ca-trust-extracted\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.273223 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.273176 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-installation-pull-secrets\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.273504 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.273487 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/12d607d3-5fcc-4357-bd03-1c9697641926-image-registry-private-configuration\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.273614 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.273599 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-registry-tls\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.277931 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.277905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-bound-sa-token\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.277931 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.277921 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6lbp\" (UniqueName: \"kubernetes.io/projected/12d607d3-5fcc-4357-bd03-1c9697641926-kube-api-access-z6lbp\") pod \"image-registry-6c48d6f8c8-vv7p8\" (UID: \"12d607d3-5fcc-4357-bd03-1c9697641926\") " pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.431281 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.431187 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:28.543360 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.543329 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="552d6379-2430-43c9-af3e-d93c91c16a07" path="/var/lib/kubelet/pods/552d6379-2430-43c9-af3e-d93c91c16a07/volumes" Jun 08 22:51:28.563177 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:28.563146 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c48d6f8c8-vv7p8"] Jun 08 22:51:28.565956 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:28.565932 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12d607d3_5fcc_4357_bd03_1c9697641926.slice/crio-48c6e69434bc98f8c5ecd0c523a97abf87a216686740f13623999138ff2cc7dd WatchSource:0}: Error finding container 48c6e69434bc98f8c5ecd0c523a97abf87a216686740f13623999138ff2cc7dd: Status 404 returned error can't find the container with id 48c6e69434bc98f8c5ecd0c523a97abf87a216686740f13623999138ff2cc7dd Jun 08 22:51:29.216074 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.216028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" event={"ID":"12d607d3-5fcc-4357-bd03-1c9697641926","Type":"ContainerStarted","Data":"a7ed131be71e221e2bf80fa89454d935621cf964f4e2ccf5515ccb7c9e9c7b50"} Jun 08 22:51:29.216285 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.216096 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" event={"ID":"12d607d3-5fcc-4357-bd03-1c9697641926","Type":"ContainerStarted","Data":"48c6e69434bc98f8c5ecd0c523a97abf87a216686740f13623999138ff2cc7dd"} Jun 08 22:51:29.216285 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.216132 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:51:29.233112 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.233049 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podStartSLOduration=1.233027173 podStartE2EDuration="1.233027173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:29.232373075 +0000 UTC m=+211.112280190" watchObservedRunningTime="2026-06-08 22:51:29.233027173 +0000 UTC m=+211.112934266" Jun 08 22:51:29.312864 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.312827 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rf6j8"] Jun 08 22:51:29.317403 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.317379 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.319651 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.319627 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 08 22:51:29.319651 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.319633 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 08 22:51:29.319848 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.319662 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pcscd\"" Jun 08 22:51:29.326110 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.326085 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rf6j8"] Jun 08 22:51:29.379076 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.379035 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.379076 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.379079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-data-volume\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.379303 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.379141 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-crio-socket\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.379303 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.379194 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pppn8\" (UniqueName: \"kubernetes.io/projected/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-api-access-pppn8\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.379303 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.379223 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.479846 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.479762 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.479846 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.479802 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-data-volume\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.479846 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.479830 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-crio-socket\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.480010 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.479905 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-crio-socket\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.480010 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:29.479923 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:29.480010 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.479953 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pppn8\" (UniqueName: \"kubernetes.io/projected/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-api-access-pppn8\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.480010 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:29.479989 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls podName:cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa nodeName:}" failed. No retries permitted until 2026-06-08 22:51:29.979973442 +0000 UTC m=+211.859880512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rf6j8" (UID: "cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:29.480154 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.480029 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.480222 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.480203 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-data-volume\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.480521 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.480504 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.488225 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.488199 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pppn8\" (UniqueName: \"kubernetes.io/projected/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-kube-api-access-pppn8\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.984264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:29.984221 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:29.984472 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:29.984371 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:29.984472 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:29.984428 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls podName:cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa nodeName:}" failed. No retries permitted until 2026-06-08 22:51:30.984414461 +0000 UTC m=+212.864321531 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rf6j8" (UID: "cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:30.992826 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:30.992775 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:30.993235 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:30.992933 2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 08 22:51:30.993235 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:30.993012 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls podName:cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa nodeName:}" failed. No retries permitted until 2026-06-08 22:51:32.992996384 +0000 UTC m=+214.872903454 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-rf6j8" (UID: "cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa") : secret "insights-runtime-extractor-tls" not found Jun 08 22:51:33.009429 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:33.009385 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:33.011915 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:33.011894 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rf6j8\" (UID: \"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa\") " pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:33.227166 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:33.227132 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rf6j8" Jun 08 22:51:33.360211 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:33.360171 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rf6j8"] Jun 08 22:51:33.363178 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:33.363151 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdc44ca0_0f2a_48a6_afb4_f0367a1d06aa.slice/crio-11f19947bfa904053fb958b76d174f59df329e59665ca6fa1be2622c1558f74a WatchSource:0}: Error finding container 11f19947bfa904053fb958b76d174f59df329e59665ca6fa1be2622c1558f74a: Status 404 returned error can't find the container with id 11f19947bfa904053fb958b76d174f59df329e59665ca6fa1be2622c1558f74a Jun 08 22:51:34.233076 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:34.233037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rf6j8" event={"ID":"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa","Type":"ContainerStarted","Data":"fdcc5551c7ebbea58f6c02ff40996b41194db395b18a05bbd81fbd1c0da6ffe4"} Jun 08 22:51:34.233473 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:34.233095 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rf6j8" event={"ID":"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa","Type":"ContainerStarted","Data":"4d77c81cffd26e57ab9d74396209654c9b8a18a7491c651889ca54cf092b6869"} Jun 08 22:51:34.233473 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:34.233113 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rf6j8" event={"ID":"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa","Type":"ContainerStarted","Data":"11f19947bfa904053fb958b76d174f59df329e59665ca6fa1be2622c1558f74a"} Jun 08 22:51:36.241460 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:36.241414 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rf6j8" event={"ID":"cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa","Type":"ContainerStarted","Data":"fdad6094a07330362cb40a9c64a7cf44560387f5061fdb928ed90a268527197d"} Jun 08 22:51:36.258137 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:36.258081 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rf6j8" podStartSLOduration=5.202878086 podStartE2EDuration="7.258060396s" totalImagesPullingTime="2.05518231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 22:51:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:36.256844673 +0000 UTC m=+218.136751764" watchObservedRunningTime="2026-06-08 22:51:36.258060396 +0000 UTC m=+218.137967487" Jun 08 22:51:43.695280 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:43.695223 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:43.695890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:43.695870 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5148df69-7982-4755-bbe5-6f131cf83393-service-ca-bundle\") pod \"router-default-79cdcd997f-qf4wj\" (UID: \"5148df69-7982-4755-bbe5-6f131cf83393\") " pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:43.865360 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:43.865313 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:43.998936 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:43.998898 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-79cdcd997f-qf4wj"] Jun 08 22:51:44.001718 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:44.001687 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5148df69_7982_4755_bbe5_6f131cf83393.slice/crio-e77b69c85dc9bc0e1ee3762c1b15ffb3571580efb1f09dbaa3798a16d2bb7402 WatchSource:0}: Error finding container e77b69c85dc9bc0e1ee3762c1b15ffb3571580efb1f09dbaa3798a16d2bb7402: Status 404 returned error can't find the container with id e77b69c85dc9bc0e1ee3762c1b15ffb3571580efb1f09dbaa3798a16d2bb7402 Jun 08 22:51:44.271764 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:44.271674 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" event={"ID":"5148df69-7982-4755-bbe5-6f131cf83393","Type":"ContainerStarted","Data":"eed4ce1ef1aad6e26c5aa1987e12e3364f5c983d58444c37b3904d9ac44faed7"} Jun 08 22:51:44.271764 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:44.271751 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" event={"ID":"5148df69-7982-4755-bbe5-6f131cf83393","Type":"ContainerStarted","Data":"e77b69c85dc9bc0e1ee3762c1b15ffb3571580efb1f09dbaa3798a16d2bb7402"} Jun 08 22:51:44.289634 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:44.289580 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" podStartSLOduration=33.28956427 podStartE2EDuration="33.28956427s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:44.288838161 +0000 UTC m=+226.168745252" watchObservedRunningTime="2026-06-08 22:51:44.28956427 +0000 UTC m=+226.169471340" Jun 08 22:51:44.865676 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:44.865640 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:44.868971 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:44.868912 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:45.275289 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:45.275208 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:45.276590 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:45.276570 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-79cdcd997f-qf4wj" Jun 08 22:51:48.435693 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:48.435657 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:48.436063 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:48.435715 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:50.223949 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:50.223908 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:50.224385 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:50.223981 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:54.276578 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.276537 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:51:54.282416 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.282390 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.286352 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.286326 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 08 22:51:54.286566 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.286543 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 08 22:51:54.286626 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.286570 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 08 22:51:54.292617 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.292589 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 08 22:51:54.297038 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.297017 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l2wd6\"" Jun 08 22:51:54.297657 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.297640 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 08 22:51:54.297788 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.297773 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 08 22:51:54.298065 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.298046 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 08 22:51:54.298468 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.298430 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 08 22:51:54.313230 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.313202 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:51:54.384254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384217 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384254 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384277 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqhsl\" (UniqueName: \"kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384378 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384419 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384484 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.384669 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.384590 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.485760 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.485700 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dqhsl\" (UniqueName: \"kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.485760 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.485762 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.485995 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.485935 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.485995 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.485977 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486087 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486051 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486087 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486077 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486186 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486102 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486623 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486571 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486801 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486781 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486801 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486792 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.486901 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.486807 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.489164 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.489142 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.489276 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.489252 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.493538 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.493514 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqhsl\" (UniqueName: \"kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl\") pod \"console-8f9bc6d5f-lxllx\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.593201 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.593158 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:51:54.732753 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:54.732717 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:51:54.735625 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:54.735588 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09a5728c_4d4a_48f4_ad54_1d8334bb242a.slice/crio-41867102bbfa07697021c42964389bb92b01403ba9278f9551d3268831378fe6 WatchSource:0}: Error finding container 41867102bbfa07697021c42964389bb92b01403ba9278f9551d3268831378fe6: Status 404 returned error can't find the container with id 41867102bbfa07697021c42964389bb92b01403ba9278f9551d3268831378fe6 Jun 08 22:51:55.309526 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:55.309462 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f9bc6d5f-lxllx" event={"ID":"09a5728c-4d4a-48f4-ad54-1d8334bb242a","Type":"ContainerStarted","Data":"41867102bbfa07697021c42964389bb92b01403ba9278f9551d3268831378fe6"} Jun 08 22:51:56.811009 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.810964 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-qctrf"] Jun 08 22:51:56.818494 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.818373 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:56.821212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.821187 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 08 22:51:56.821358 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.821260 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 08 22:51:56.824148 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.822761 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 08 22:51:56.824148 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.822830 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 08 22:51:56.824148 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.822766 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bl99s\"" Jun 08 22:51:56.824462 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.824421 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 08 22:51:56.825767 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.825747 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-qctrf"] Jun 08 22:51:56.906098 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.906055 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:56.906294 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.906142 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:56.906294 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.906223 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lz4z4\" (UniqueName: \"kubernetes.io/projected/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-kube-api-access-lz4z4\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:56.906294 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:56.906267 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.006881 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.006837 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.007074 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.006971 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lz4z4\" (UniqueName: \"kubernetes.io/projected/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-kube-api-access-lz4z4\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.007074 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.007010 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.007189 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.007080 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.007247 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:57.007230 2583 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 08 22:51:57.007302 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:51:57.007285 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls podName:7e2f567e-4642-4d32-9e78-deffc1e3e3c5 nodeName:}" failed. No retries permitted until 2026-06-08 22:51:57.507266612 +0000 UTC m=+239.387173685 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls") pod "prometheus-operator-7f7d445d84-qctrf" (UID: "7e2f567e-4642-4d32-9e78-deffc1e3e3c5") : secret "prometheus-operator-tls" not found Jun 08 22:51:57.007643 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.007609 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-metrics-client-ca\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.010350 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.010321 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.016393 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.016343 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lz4z4\" (UniqueName: \"kubernetes.io/projected/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-kube-api-access-lz4z4\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.512571 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.512527 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.515220 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.515197 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7e2f567e-4642-4d32-9e78-deffc1e3e3c5-prometheus-operator-tls\") pod \"prometheus-operator-7f7d445d84-qctrf\" (UID: \"7e2f567e-4642-4d32-9e78-deffc1e3e3c5\") " pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.732077 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.732035 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" Jun 08 22:51:57.886156 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:57.886117 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7f7d445d84-qctrf"] Jun 08 22:51:57.888613 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:51:57.888579 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e2f567e_4642_4d32_9e78_deffc1e3e3c5.slice/crio-1046600636df23bde1b60d34ba8d644b56980a15ceb456c23a93b28311547296 WatchSource:0}: Error finding container 1046600636df23bde1b60d34ba8d644b56980a15ceb456c23a93b28311547296: Status 404 returned error can't find the container with id 1046600636df23bde1b60d34ba8d644b56980a15ceb456c23a93b28311547296 Jun 08 22:51:58.322300 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:58.322259 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f9bc6d5f-lxllx" event={"ID":"09a5728c-4d4a-48f4-ad54-1d8334bb242a","Type":"ContainerStarted","Data":"1a3716e8e891c6eef95720246de4eb2c33969c1808235c58749439972f8941bc"} Jun 08 22:51:58.323391 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:58.323365 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" event={"ID":"7e2f567e-4642-4d32-9e78-deffc1e3e3c5","Type":"ContainerStarted","Data":"1046600636df23bde1b60d34ba8d644b56980a15ceb456c23a93b28311547296"} Jun 08 22:51:58.339959 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:58.339895 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8f9bc6d5f-lxllx" podStartSLOduration=1.8091011209999999 podStartE2EDuration="4.339876885s" totalImagesPullingTime="2.530775764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:58.338028214 +0000 UTC m=+240.217935307" watchObservedRunningTime="2026-06-08 22:51:58.339876885 +0000 UTC m=+240.219783977" Jun 08 22:51:58.437301 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:58.437258 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:51:58.437515 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:58.437332 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:51:59.328789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:59.328746 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" event={"ID":"7e2f567e-4642-4d32-9e78-deffc1e3e3c5","Type":"ContainerStarted","Data":"7fe3c43a5accecd3dd108c5b094baf13c32ff415731b8f81a4ccd046e0a730e0"} Jun 08 22:51:59.329269 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:59.328848 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" event={"ID":"7e2f567e-4642-4d32-9e78-deffc1e3e3c5","Type":"ContainerStarted","Data":"f6fee463a2333b27a8612ae14db19ebe398a35b12952447fe1695e459f858f56"} Jun 08 22:51:59.347171 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:51:59.347120 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7f7d445d84-qctrf" podStartSLOduration=2.217900725 podStartE2EDuration="3.347104397s" totalImagesPullingTime="1.129203672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:51:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:51:59.345161139 +0000 UTC m=+241.225068230" watchObservedRunningTime="2026-06-08 22:51:59.347104397 +0000 UTC m=+241.227011489" Jun 08 22:52:00.224163 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:00.224127 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:00.224347 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:00.224184 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:01.189873 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.189838 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4f5h9"] Jun 08 22:52:01.193960 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.193930 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.196945 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.196575 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 08 22:52:01.196945 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.196661 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ksq6q\"" Jun 08 22:52:01.196945 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.196888 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 08 22:52:01.197201 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.197168 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 08 22:52:01.347528 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347493 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-root\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347722 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347538 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-tls\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347722 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347614 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gg9rb\" (UniqueName: \"kubernetes.io/projected/f439438b-d0a7-4d62-bda9-f200d5cc2017-kube-api-access-gg9rb\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347722 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347679 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-sys\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347722 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347705 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347943 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347781 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-textfile\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347943 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347835 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-metrics-client-ca\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347943 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347879 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-wtmp\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.347943 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.347920 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-accelerators-collector-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449113 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-metrics-client-ca\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449113 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449065 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-wtmp\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449201 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-wtmp\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449210 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-accelerators-collector-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449263 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-root\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449289 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-tls\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449337 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gg9rb\" (UniqueName: \"kubernetes.io/projected/f439438b-d0a7-4d62-bda9-f200d5cc2017-kube-api-access-gg9rb\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449624 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449394 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-root\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449624 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449407 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-sys\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449624 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449479 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f439438b-d0a7-4d62-bda9-f200d5cc2017-sys\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449624 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449499 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449624 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449545 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-textfile\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449854 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449827 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-accelerators-collector-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449913 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449827 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f439438b-d0a7-4d62-bda9-f200d5cc2017-metrics-client-ca\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.449913 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.449875 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-textfile\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.452076 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.452048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.452221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.452197 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f439438b-d0a7-4d62-bda9-f200d5cc2017-node-exporter-tls\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.457516 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.457496 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gg9rb\" (UniqueName: \"kubernetes.io/projected/f439438b-d0a7-4d62-bda9-f200d5cc2017-kube-api-access-gg9rb\") pod \"node-exporter-4f5h9\" (UID: \"f439438b-d0a7-4d62-bda9-f200d5cc2017\") " pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.506525 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:01.506489 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4f5h9" Jun 08 22:52:01.518893 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:52:01.518858 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf439438b_d0a7_4d62_bda9_f200d5cc2017.slice/crio-f25fdc807dcc854674585eef9258c0057fe5586186f7c62bacb02d13a22fd661 WatchSource:0}: Error finding container f25fdc807dcc854674585eef9258c0057fe5586186f7c62bacb02d13a22fd661: Status 404 returned error can't find the container with id f25fdc807dcc854674585eef9258c0057fe5586186f7c62bacb02d13a22fd661 Jun 08 22:52:02.341323 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:02.341278 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4f5h9" event={"ID":"f439438b-d0a7-4d62-bda9-f200d5cc2017","Type":"ContainerStarted","Data":"f25fdc807dcc854674585eef9258c0057fe5586186f7c62bacb02d13a22fd661"} Jun 08 22:52:03.345658 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:03.345621 2583 generic.go:356] "Generic (PLEG): container finished" podID="f439438b-d0a7-4d62-bda9-f200d5cc2017" containerID="615df3a642e3ee652793d4c99d826f29abae136d377f19577fa6b5a9f5da2f99" exitCode=0 Jun 08 22:52:03.346102 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:03.345712 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4f5h9" event={"ID":"f439438b-d0a7-4d62-bda9-f200d5cc2017","Type":"ContainerDied","Data":"615df3a642e3ee652793d4c99d826f29abae136d377f19577fa6b5a9f5da2f99"} Jun 08 22:52:04.351283 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.351238 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4f5h9" event={"ID":"f439438b-d0a7-4d62-bda9-f200d5cc2017","Type":"ContainerStarted","Data":"34858d48e33e51c22173a7448f8dfbde5d0921c9732713899d630d878ff5ae22"} Jun 08 22:52:04.351757 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.351301 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4f5h9" event={"ID":"f439438b-d0a7-4d62-bda9-f200d5cc2017","Type":"ContainerStarted","Data":"4701bab5e354d83436b3249546ed48a0408450a78a724c13871d613e0cc4ae54"} Jun 08 22:52:04.369458 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.369396 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4f5h9" podStartSLOduration=2.298771794 podStartE2EDuration="3.369377304s" totalImagesPullingTime="993.933072ms" totalInitContainerRuntime="76.672438ms" isStatefulPod=true podCreationTimestamp="2026-06-08 22:52:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:04.368281632 +0000 UTC m=+246.248188724" watchObservedRunningTime="2026-06-08 22:52:04.369377304 +0000 UTC m=+246.249284396" Jun 08 22:52:04.593572 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.593478 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:52:04.593572 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.593537 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:52:04.594954 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.594929 2583 patch_prober.go:28] interesting pod/console-8f9bc6d5f-lxllx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jun 08 22:52:04.595052 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:04.594976 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jun 08 22:52:08.436206 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.436163 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:08.436741 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.436229 2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:08.436741 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.436282 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:52:08.436943 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.436902 2583 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"a7ed131be71e221e2bf80fa89454d935621cf964f4e2ccf5515ccb7c9e9c7b50"} pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" containerMessage="Container registry failed liveness probe, will be restarted" Jun 08 22:52:08.440222 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.440183 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:08.440358 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:08.440231 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:10.434188 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:10.434146 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:52:10.436702 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:10.436681 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c-metrics-certs\") pod \"network-metrics-daemon-lw8qj\" (UID: \"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c\") " pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:52:10.642394 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:10.642359 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wjrnx\"" Jun 08 22:52:10.650397 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:10.650370 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lw8qj" Jun 08 22:52:10.791763 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:10.791727 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lw8qj"] Jun 08 22:52:10.794832 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:52:10.794796 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf1db79_d3a9_4b48_bc18_e8d3fcaa035c.slice/crio-5dbfe14d8d73366ab051c91a9eb4a046cee3bd617a0485d3eed2d4418831f149 WatchSource:0}: Error finding container 5dbfe14d8d73366ab051c91a9eb4a046cee3bd617a0485d3eed2d4418831f149: Status 404 returned error can't find the container with id 5dbfe14d8d73366ab051c91a9eb4a046cee3bd617a0485d3eed2d4418831f149 Jun 08 22:52:11.375940 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:11.375907 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw8qj" event={"ID":"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c","Type":"ContainerStarted","Data":"5dbfe14d8d73366ab051c91a9eb4a046cee3bd617a0485d3eed2d4418831f149"} Jun 08 22:52:12.381913 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:12.381870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw8qj" event={"ID":"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c","Type":"ContainerStarted","Data":"bba48d832e9ef427a859ede5aec2920517e450068c236f240672ce809f9838a4"} Jun 08 22:52:12.382387 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:12.381939 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lw8qj" event={"ID":"cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c","Type":"ContainerStarted","Data":"756d6290ffb9fba681d05c13073ef074e5849d018cbb4294aceef2ffe204eb20"} Jun 08 22:52:12.398237 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:12.398180 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lw8qj" podStartSLOduration=253.603132829 podStartE2EDuration="4m14.398162562s" totalImagesPullingTime="795.029733ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:47:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:12.397540603 +0000 UTC m=+254.277447706" watchObservedRunningTime="2026-06-08 22:52:12.398162562 +0000 UTC m=+254.278069690" Jun 08 22:52:14.594530 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:14.594488 2583 patch_prober.go:28] interesting pod/console-8f9bc6d5f-lxllx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jun 08 22:52:14.594954 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:14.594546 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jun 08 22:52:18.440831 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:18.440793 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:18.441206 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:18.440849 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:24.594590 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:24.594544 2583 patch_prober.go:28] interesting pod/console-8f9bc6d5f-lxllx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jun 08 22:52:24.594994 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:24.594607 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jun 08 22:52:28.440958 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.440924 2583 patch_prober.go:28] interesting pod/image-registry-6c48d6f8c8-vv7p8 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 08 22:52:28.441329 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.440987 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:52:28.978792 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.978757 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-wm68d"] Jun 08 22:52:28.983730 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.983704 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:28.987765 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.987742 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-46snp\"" Jun 08 22:52:28.987765 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.987742 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 08 22:52:28.990979 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:28.990953 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-wm68d"] Jun 08 22:52:29.066178 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.066147 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:52:29.069280 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.069261 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.079964 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.079936 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:52:29.080221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.080203 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxk5c\" (UniqueName: \"kubernetes.io/projected/b4c6411d-5041-46d9-b4e1-5a03d5528b77-kube-api-access-sxk5c\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.080281 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.080233 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4c6411d-5041-46d9-b4e1-5a03d5528b77-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.181692 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181650 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.181692 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181706 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxk5c\" (UniqueName: \"kubernetes.io/projected/b4c6411d-5041-46d9-b4e1-5a03d5528b77-kube-api-access-sxk5c\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.181966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181727 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.181966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181789 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4c6411d-5041-46d9-b4e1-5a03d5528b77-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.181966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181842 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x674t\" (UniqueName: \"kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.181966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181880 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.181966 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181898 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.182179 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.181967 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.182179 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.182013 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.184640 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.184613 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b4c6411d-5041-46d9-b4e1-5a03d5528b77-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.190666 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.190636 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxk5c\" (UniqueName: \"kubernetes.io/projected/b4c6411d-5041-46d9-b4e1-5a03d5528b77-kube-api-access-sxk5c\") pod \"monitoring-plugin-76b68cc874-wm68d\" (UID: \"b4c6411d-5041-46d9-b4e1-5a03d5528b77\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.283241 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283138 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283241 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283197 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283241 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283231 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283560 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283254 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x674t\" (UniqueName: \"kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283560 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283290 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283560 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283312 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.283560 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.283349 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.284079 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.284048 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.284193 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.284163 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.284691 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.284651 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.284870 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.284844 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.286839 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.286503 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.286839 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.286795 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.290732 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.290688 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x674t\" (UniqueName: \"kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t\") pod \"console-7b996f986c-4gr8p\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.294544 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.294521 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:29.377642 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.377609 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:29.460090 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.460052 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-wm68d"] Jun 08 22:52:29.464593 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:52:29.464562 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c6411d_5041_46d9_b4e1_5a03d5528b77.slice/crio-5221c1442a817f371c37c65504be9ef2d4bb66cfdddab96721329ad3c6598717 WatchSource:0}: Error finding container 5221c1442a817f371c37c65504be9ef2d4bb66cfdddab96721329ad3c6598717: Status 404 returned error can't find the container with id 5221c1442a817f371c37c65504be9ef2d4bb66cfdddab96721329ad3c6598717 Jun 08 22:52:29.539000 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:29.538870 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:52:29.542293 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:52:29.542255 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod834afc02_2eb9_4c7b_9e68_1cd3cf5d401e.slice/crio-85bd2831b7ad2dd8f3e4323608c30af06ffdc43ee9e1b0a9a846a7dd86b22f21 WatchSource:0}: Error finding container 85bd2831b7ad2dd8f3e4323608c30af06ffdc43ee9e1b0a9a846a7dd86b22f21: Status 404 returned error can't find the container with id 85bd2831b7ad2dd8f3e4323608c30af06ffdc43ee9e1b0a9a846a7dd86b22f21 Jun 08 22:52:30.449201 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:30.449166 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b996f986c-4gr8p" event={"ID":"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e","Type":"ContainerStarted","Data":"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf"} Jun 08 22:52:30.449390 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:30.449218 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b996f986c-4gr8p" event={"ID":"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e","Type":"ContainerStarted","Data":"85bd2831b7ad2dd8f3e4323608c30af06ffdc43ee9e1b0a9a846a7dd86b22f21"} Jun 08 22:52:30.450368 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:30.450347 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" event={"ID":"b4c6411d-5041-46d9-b4e1-5a03d5528b77","Type":"ContainerStarted","Data":"5221c1442a817f371c37c65504be9ef2d4bb66cfdddab96721329ad3c6598717"} Jun 08 22:52:30.465378 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:30.465319 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b996f986c-4gr8p" podStartSLOduration=1.465297638 podStartE2EDuration="1.465297638s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:52:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:30.464387598 +0000 UTC m=+272.344294703" watchObservedRunningTime="2026-06-08 22:52:30.465297638 +0000 UTC m=+272.345204726" Jun 08 22:52:31.456862 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:31.456814 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" event={"ID":"b4c6411d-5041-46d9-b4e1-5a03d5528b77","Type":"ContainerStarted","Data":"15013a4a6808bc6ece95abefb652c732a06630ab16fa0cf08ac59acd73b79437"} Jun 08 22:52:31.457215 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:31.457196 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:31.461642 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:31.461616 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" Jun 08 22:52:31.470739 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:31.470699 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-wm68d" podStartSLOduration=1.900016939 podStartE2EDuration="3.470686425s" totalImagesPullingTime="1.570669486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:52:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:31.4702443 +0000 UTC m=+273.350151393" watchObservedRunningTime="2026-06-08 22:52:31.470686425 +0000 UTC m=+273.350593524" Jun 08 22:52:33.456818 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:33.456768 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" podUID="12d607d3-5fcc-4357-bd03-1c9697641926" containerName="registry" containerID="cri-o://a7ed131be71e221e2bf80fa89454d935621cf964f4e2ccf5515ccb7c9e9c7b50" gracePeriod=30 Jun 08 22:52:34.469546 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.469515 2583 generic.go:356] "Generic (PLEG): container finished" podID="12d607d3-5fcc-4357-bd03-1c9697641926" containerID="a7ed131be71e221e2bf80fa89454d935621cf964f4e2ccf5515ccb7c9e9c7b50" exitCode=0 Jun 08 22:52:34.469988 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.469572 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" event={"ID":"12d607d3-5fcc-4357-bd03-1c9697641926","Type":"ContainerDied","Data":"a7ed131be71e221e2bf80fa89454d935621cf964f4e2ccf5515ccb7c9e9c7b50"} Jun 08 22:52:34.469988 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.469612 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" event={"ID":"12d607d3-5fcc-4357-bd03-1c9697641926","Type":"ContainerStarted","Data":"aa9fdb55902dda2178cba862fb297f9f267a2bfc28e87e1f2f4e6a5d270c09de"} Jun 08 22:52:34.469988 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.469783 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:52:34.593697 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.593665 2583 patch_prober.go:28] interesting pod/console-8f9bc6d5f-lxllx container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jun 08 22:52:34.593847 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:34.593732 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jun 08 22:52:36.478333 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:36.478297 2583 generic.go:356] "Generic (PLEG): container finished" podID="0df19849-731a-4b98-b3a4-26739a64e434" containerID="efb9e5aa697b5b7fbfca0abd8c9c8071f4c83d7d5edba34dc2b095c6c820868a" exitCode=0 Jun 08 22:52:36.478758 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:36.478371 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hmw8f" event={"ID":"0df19849-731a-4b98-b3a4-26739a64e434","Type":"ContainerDied","Data":"efb9e5aa697b5b7fbfca0abd8c9c8071f4c83d7d5edba34dc2b095c6c820868a"} Jun 08 22:52:36.478758 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:36.478734 2583 scope.go:117] "RemoveContainer" containerID="efb9e5aa697b5b7fbfca0abd8c9c8071f4c83d7d5edba34dc2b095c6c820868a" Jun 08 22:52:37.483998 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:37.483960 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-896549976-hmw8f" event={"ID":"0df19849-731a-4b98-b3a4-26739a64e434","Type":"ContainerStarted","Data":"40d987059ce26e4d033872eb339d50ede01def398914aa4acb38a535a650105c"} Jun 08 22:52:39.378171 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:39.378125 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:39.378643 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:39.378189 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:39.379868 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:39.379839 2583 patch_prober.go:28] interesting pod/console-7b996f986c-4gr8p container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused" start-of-body= Jun 08 22:52:39.380006 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:39.379893 2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7b996f986c-4gr8p" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerName="console" probeResult="failure" output="Get \"https://10.133.0.19:8443/health\": dial tcp 10.133.0.19:8443: connect: connection refused" Jun 08 22:52:40.050427 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:52:40.050383 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vvvh2" podUID="411ad465-05c7-4a35-8151-b92c1ac48a1e" Jun 08 22:52:40.496130 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:40.496047 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:41.564094 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:41.564054 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-wm68d_b4c6411d-5041-46d9-b4e1-5a03d5528b77/monitoring-plugin/0.log" Jun 08 22:52:41.764813 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:41.764785 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/init-textfile/0.log" Jun 08 22:52:41.965105 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:41.965021 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/node-exporter/0.log" Jun 08 22:52:42.163931 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:42.163894 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/kube-rbac-proxy/0.log" Jun 08 22:52:43.509391 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:43.509354 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:43.511961 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:43.511936 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/411ad465-05c7-4a35-8151-b92c1ac48a1e-metrics-tls\") pod \"dns-default-vvvh2\" (UID: \"411ad465-05c7-4a35-8151-b92c1ac48a1e\") " pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:43.799225 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:43.799187 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jnlbh\"" Jun 08 22:52:43.807293 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:43.807265 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:43.953563 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:43.949084 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vvvh2"] Jun 08 22:52:43.955778 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:52:43.955741 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod411ad465_05c7_4a35_8151_b92c1ac48a1e.slice/crio-e7f5f128a43b298ea5a057ed05725c2d46d8834c54ed85f4b33ef352fbbd335a WatchSource:0}: Error finding container e7f5f128a43b298ea5a057ed05725c2d46d8834c54ed85f4b33ef352fbbd335a: Status 404 returned error can't find the container with id e7f5f128a43b298ea5a057ed05725c2d46d8834c54ed85f4b33ef352fbbd335a Jun 08 22:52:44.513223 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:44.513175 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvvh2" event={"ID":"411ad465-05c7-4a35-8151-b92c1ac48a1e","Type":"ContainerStarted","Data":"e7f5f128a43b298ea5a057ed05725c2d46d8834c54ed85f4b33ef352fbbd335a"} Jun 08 22:52:44.599282 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:44.599246 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:52:44.604094 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:44.604063 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:52:45.567516 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:45.567481 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-qctrf_7e2f567e-4642-4d32-9e78-deffc1e3e3c5/prometheus-operator/0.log" Jun 08 22:52:45.765057 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:45.764969 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-qctrf_7e2f567e-4642-4d32-9e78-deffc1e3e3c5/kube-rbac-proxy/0.log" Jun 08 22:52:46.523397 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:46.523362 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvvh2" event={"ID":"411ad465-05c7-4a35-8151-b92c1ac48a1e","Type":"ContainerStarted","Data":"d410a0dba228bb7fef8035723393704547c46e02844773a04782968e0c928fdd"} Jun 08 22:52:46.523607 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:46.523412 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vvvh2" event={"ID":"411ad465-05c7-4a35-8151-b92c1ac48a1e","Type":"ContainerStarted","Data":"97dac17fd6f5f5ae57cba8057f821e78a52cc06daddd11d66c0da8464f80ee2b"} Jun 08 22:52:46.523607 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:46.523486 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:46.539365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:46.539312 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-vvvh2" podStartSLOduration=252.051547681 podStartE2EDuration="4m13.539293261s" totalImagesPullingTime="1.48774558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:52:46.538163752 +0000 UTC m=+288.418070844" watchObservedRunningTime="2026-06-08 22:52:46.539293261 +0000 UTC m=+288.419200352" Jun 08 22:52:48.165071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:48.165038 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7b996f986c-4gr8p_834afc02-2eb9-4c7b-9e68-1cd3cf5d401e/console/0.log" Jun 08 22:52:48.365132 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:48.365102 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8f9bc6d5f-lxllx_09a5728c-4d4a-48f4-ad54-1d8334bb242a/console/0.log" Jun 08 22:52:49.382301 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:49.382262 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:49.386125 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:49.386103 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:52:49.442855 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:49.442823 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:52:49.564887 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:49.564857 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvvh2_411ad465-05c7-4a35-8151-b92c1ac48a1e/dns/0.log" Jun 08 22:52:49.765307 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:49.765226 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvvh2_411ad465-05c7-4a35-8151-b92c1ac48a1e/kube-rbac-proxy/0.log" Jun 08 22:52:50.364684 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:50.364656 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vk4sh_d70d3927-89dd-457d-a77c-2d4c4bdaf48a/dns-node-resolver/0.log" Jun 08 22:52:50.565718 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:50.565689 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c48d6f8c8-vv7p8_12d607d3-5fcc-4357-bd03-1c9697641926/registry/0.log" Jun 08 22:52:50.764965 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:50.764885 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c48d6f8c8-vv7p8_12d607d3-5fcc-4357-bd03-1c9697641926/registry/1.log" Jun 08 22:52:51.764656 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:51.764631 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lcf6r_7948a206-759b-4031-a1bc-af2b2e94ec55/node-ca/0.log" Jun 08 22:52:51.964842 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:51.964815 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log" Jun 08 22:52:52.165006 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:52.164977 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-94b4v_f91f94b7-610c-4949-8a36-e3cf1f35c9d6/serve-healthcheck-canary/0.log" Jun 08 22:52:55.477018 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:55.476993 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c48d6f8c8-vv7p8" Jun 08 22:52:56.529861 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:56.529829 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vvvh2" Jun 08 22:52:58.492259 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:58.492232 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:52:58.495264 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:58.495240 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:52:58.496355 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:52:58.496326 2583 kubelet.go:1648] "Image garbage collection succeeded" Jun 08 22:53:14.463543 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.463493 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" containerID="cri-o://1a3716e8e891c6eef95720246de4eb2c33969c1808235c58749439972f8941bc" gracePeriod=15 Jun 08 22:53:14.600789 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.600751 2583 patch_prober.go:28] interesting pod/console-8f9bc6d5f-lxllx container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" start-of-body= Jun 08 22:53:14.600978 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.600818 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-8f9bc6d5f-lxllx" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" probeResult="failure" output="Get \"https://10.133.0.16:8443/health\": dial tcp 10.133.0.16:8443: connect: connection refused" Jun 08 22:53:14.646781 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.646744 2583 generic.go:356] "Generic (PLEG): container finished" podID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerID="1a3716e8e891c6eef95720246de4eb2c33969c1808235c58749439972f8941bc" exitCode=0 Jun 08 22:53:14.646781 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.646766 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f9bc6d5f-lxllx" event={"ID":"09a5728c-4d4a-48f4-ad54-1d8334bb242a","Type":"ContainerDied","Data":"1a3716e8e891c6eef95720246de4eb2c33969c1808235c58749439972f8941bc"} Jun 08 22:53:14.720672 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.720606 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:53:14.783430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783396 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783484 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783524 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783553 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783582 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqhsl\" (UniqueName: \"kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783964 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783604 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.783964 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783651 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config\") pod \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\" (UID: \"09a5728c-4d4a-48f4-ad54-1d8334bb242a\") " Jun 08 22:53:14.784063 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.783996 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:14.784063 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.784027 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca" (OuterVolumeSpecName: "service-ca") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:14.784171 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.784057 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config" (OuterVolumeSpecName: "console-config") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:14.784279 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.784258 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:53:14.786103 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.786074 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:53:14.786254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.786133 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:53:14.786254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.786149 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl" (OuterVolumeSpecName: "kube-api-access-dqhsl") pod "09a5728c-4d4a-48f4-ad54-1d8334bb242a" (UID: "09a5728c-4d4a-48f4-ad54-1d8334bb242a"). InnerVolumeSpecName "kube-api-access-dqhsl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:53:14.884828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884771 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.884828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884822 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.884828 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884837 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.885077 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884850 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqhsl\" (UniqueName: \"kubernetes.io/projected/09a5728c-4d4a-48f4-ad54-1d8334bb242a-kube-api-access-dqhsl\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.885077 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884862 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.885077 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884874 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09a5728c-4d4a-48f4-ad54-1d8334bb242a-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:14.885077 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:14.884885 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09a5728c-4d4a-48f4-ad54-1d8334bb242a-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:53:15.652002 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:15.651969 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8f9bc6d5f-lxllx" Jun 08 22:53:15.652002 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:15.651986 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8f9bc6d5f-lxllx" event={"ID":"09a5728c-4d4a-48f4-ad54-1d8334bb242a","Type":"ContainerDied","Data":"41867102bbfa07697021c42964389bb92b01403ba9278f9551d3268831378fe6"} Jun 08 22:53:15.652533 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:15.652055 2583 scope.go:117] "RemoveContainer" containerID="1a3716e8e891c6eef95720246de4eb2c33969c1808235c58749439972f8941bc" Jun 08 22:53:15.673188 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:15.673156 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:53:15.676502 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:15.676474 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8f9bc6d5f-lxllx"] Jun 08 22:53:16.544166 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:16.544133 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" path="/var/lib/kubelet/pods/09a5728c-4d4a-48f4-ad54-1d8334bb242a/volumes" Jun 08 22:53:50.725474 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.725417 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:53:50.726598 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.726570 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" Jun 08 22:53:50.726750 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.726739 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" Jun 08 22:53:50.726902 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.726882 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="09a5728c-4d4a-48f4-ad54-1d8334bb242a" containerName="console" Jun 08 22:53:50.730232 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.730209 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.741377 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.741348 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:53:50.868759 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868704 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.868759 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868763 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s77tl\" (UniqueName: \"kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.868993 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868856 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.868993 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868893 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.868993 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868917 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.868993 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.868967 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.869114 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.869033 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.969845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.969788 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.969845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.969846 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970085 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.969869 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s77tl\" (UniqueName: \"kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970085 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.969905 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970188 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970075 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970188 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970131 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970188 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970177 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970709 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970757 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.970811 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970848 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.971109 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.970976 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.972631 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.972609 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.972879 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.972861 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:50.978369 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:50.978292 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s77tl\" (UniqueName: \"kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl\") pod \"console-86c9b6cd6f-2z7pn\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:51.040889 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:51.040847 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:53:51.183260 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:51.183192 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:53:51.781589 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:51.781556 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9b6cd6f-2z7pn" event={"ID":"ac127502-6058-46d8-a573-49c6a83bd7e7","Type":"ContainerStarted","Data":"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161"} Jun 08 22:53:51.781996 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:51.781609 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9b6cd6f-2z7pn" event={"ID":"ac127502-6058-46d8-a573-49c6a83bd7e7","Type":"ContainerStarted","Data":"63fb9c172fe0ea258b45be213dec72d97d14ad9baf6d1c915ea7f0364034b4a9"} Jun 08 22:53:51.798240 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:53:51.798173 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86c9b6cd6f-2z7pn" podStartSLOduration=1.7981543800000002 podStartE2EDuration="1.79815438s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:53:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:53:51.79693192 +0000 UTC m=+353.676839034" watchObservedRunningTime="2026-06-08 22:53:51.79815438 +0000 UTC m=+353.678061529" Jun 08 22:54:01.041643 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:01.041597 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:54:01.042035 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:01.041678 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:54:01.046580 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:01.046553 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:54:01.821555 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:01.821524 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:54:01.862221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:01.862182 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:54:26.884155 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:26.884095 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-7b996f986c-4gr8p" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerName="console" containerID="cri-o://0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf" gracePeriod=15 Jun 08 22:54:27.141772 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.141700 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:54:27.269590 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269556 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x674t\" (UniqueName: \"kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269590 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269593 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269624 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269647 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269714 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269731 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.269845 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.269763 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert\") pod \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\" (UID: \"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e\") " Jun 08 22:54:27.270191 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.270142 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca" (OuterVolumeSpecName: "service-ca") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:54:27.270320 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.270212 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config" (OuterVolumeSpecName: "console-config") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:54:27.270320 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.270213 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:54:27.270320 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.270251 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:54:27.272031 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.272010 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:54:27.272526 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.272496 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t" (OuterVolumeSpecName: "kube-api-access-x674t") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "kube-api-access-x674t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:54:27.272611 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.272541 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" (UID: "834afc02-2eb9-4c7b-9e68-1cd3cf5d401e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:54:27.370413 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370373 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370413 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370403 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370413 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370413 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370413 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370422 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370694 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370431 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370694 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370451 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x674t\" (UniqueName: \"kubernetes.io/projected/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-kube-api-access-x674t\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.370694 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.370460 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:54:27.916591 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.916556 2583 generic.go:356] "Generic (PLEG): container finished" podID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerID="0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf" exitCode=0 Jun 08 22:54:27.917071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.916603 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b996f986c-4gr8p" event={"ID":"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e","Type":"ContainerDied","Data":"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf"} Jun 08 22:54:27.917071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.916641 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b996f986c-4gr8p" event={"ID":"834afc02-2eb9-4c7b-9e68-1cd3cf5d401e","Type":"ContainerDied","Data":"85bd2831b7ad2dd8f3e4323608c30af06ffdc43ee9e1b0a9a846a7dd86b22f21"} Jun 08 22:54:27.917071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.916642 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b996f986c-4gr8p" Jun 08 22:54:27.917071 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.916659 2583 scope.go:117] "RemoveContainer" containerID="0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf" Jun 08 22:54:27.925210 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.925194 2583 scope.go:117] "RemoveContainer" containerID="0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf" Jun 08 22:54:27.925564 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:54:27.925535 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf\": container with ID starting with 0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf not found: ID does not exist" containerID="0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf" Jun 08 22:54:27.925622 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.925578 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf"} err="failed to get container status \"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf\": rpc error: code = NotFound desc = could not find container \"0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf\": container with ID starting with 0ae7190b8bd9276bd326f092f98c36ba814d5ba1b536b0978e309c5bbf91c8bf not found: ID does not exist" Jun 08 22:54:27.936833 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.936800 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:54:27.940502 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:27.940480 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b996f986c-4gr8p"] Jun 08 22:54:28.544291 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:28.544256 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" path="/var/lib/kubelet/pods/834afc02-2eb9-4c7b-9e68-1cd3cf5d401e/volumes" Jun 08 22:54:56.808475 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.808424 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:54:56.808869 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.808692 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerName="console" Jun 08 22:54:56.808869 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.808703 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerName="console" Jun 08 22:54:56.808869 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.808783 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="834afc02-2eb9-4c7b-9e68-1cd3cf5d401e" containerName="console" Jun 08 22:54:56.811618 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.811599 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.822136 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.822106 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:54:56.919979 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.919943 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.919979 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.919981 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.920221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.920003 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.920221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.920079 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bc4j\" (UniqueName: \"kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.920221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.920110 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.920221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.920123 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:56.920221 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:56.920151 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020582 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020544 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020582 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020577 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020598 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020664 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bc4j\" (UniqueName: \"kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020698 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020719 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.020829 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.020754 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.021480 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.021406 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.021601 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.021551 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.021601 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.021576 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.022063 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.022040 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.023474 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.023432 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.023620 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.023605 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.029780 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.029756 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bc4j\" (UniqueName: \"kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j\") pod \"console-779c9bd5bc-rb2sc\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.121510 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.121386 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:54:57.258458 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:57.258410 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:54:57.261698 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:54:57.261670 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99124f96_e340_4a4d_9848_91f4c43ec607.slice/crio-bd7b90f930ed249972247c72bcd06d9bd16fa116b1eff5742164da871c088ffb WatchSource:0}: Error finding container bd7b90f930ed249972247c72bcd06d9bd16fa116b1eff5742164da871c088ffb: Status 404 returned error can't find the container with id bd7b90f930ed249972247c72bcd06d9bd16fa116b1eff5742164da871c088ffb Jun 08 22:54:58.027880 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:58.027841 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c9bd5bc-rb2sc" event={"ID":"99124f96-e340-4a4d-9848-91f4c43ec607","Type":"ContainerStarted","Data":"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16"} Jun 08 22:54:58.028430 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:58.027892 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c9bd5bc-rb2sc" event={"ID":"99124f96-e340-4a4d-9848-91f4c43ec607","Type":"ContainerStarted","Data":"bd7b90f930ed249972247c72bcd06d9bd16fa116b1eff5742164da871c088ffb"} Jun 08 22:54:58.047070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:54:58.047017 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-779c9bd5bc-rb2sc" podStartSLOduration=2.047001992 podStartE2EDuration="2.047001992s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:54:58.044681387 +0000 UTC m=+419.924588479" watchObservedRunningTime="2026-06-08 22:54:58.047001992 +0000 UTC m=+419.926909158" Jun 08 22:55:07.121867 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:07.121766 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:55:07.121867 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:07.121836 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:55:07.126938 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:07.126911 2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:55:08.070556 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:08.070524 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:55:08.114914 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:08.114879 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:55:33.142784 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.142723 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-86c9b6cd6f-2z7pn" podUID="ac127502-6058-46d8-a573-49c6a83bd7e7" containerName="console" containerID="cri-o://2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161" gracePeriod=15 Jun 08 22:55:33.395946 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.395875 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:55:33.509343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509286 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509343 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509347 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509606 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509393 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s77tl\" (UniqueName: \"kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509606 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509506 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509606 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509545 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509606 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509585 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509796 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509615 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca\") pod \"ac127502-6058-46d8-a573-49c6a83bd7e7\" (UID: \"ac127502-6058-46d8-a573-49c6a83bd7e7\") " Jun 08 22:55:33.509855 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509792 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config" (OuterVolumeSpecName: "console-config") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:55:33.509928 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.509900 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.510070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.510041 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:55:33.510070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.510051 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:55:33.510363 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.510074 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca" (OuterVolumeSpecName: "service-ca") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:55:33.511734 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.511706 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:55:33.511867 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.511840 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:55:33.511867 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.511857 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl" (OuterVolumeSpecName: "kube-api-access-s77tl") pod "ac127502-6058-46d8-a573-49c6a83bd7e7" (UID: "ac127502-6058-46d8-a573-49c6a83bd7e7"). InnerVolumeSpecName "kube-api-access-s77tl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:55:33.611008 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.610952 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.611008 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.610999 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s77tl\" (UniqueName: \"kubernetes.io/projected/ac127502-6058-46d8-a573-49c6a83bd7e7-kube-api-access-s77tl\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.611008 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.611014 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ac127502-6058-46d8-a573-49c6a83bd7e7-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.611008 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.611026 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.611279 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.611038 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:33.611279 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:33.611050 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ac127502-6058-46d8-a573-49c6a83bd7e7-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:55:34.162841 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.162800 2583 generic.go:356] "Generic (PLEG): container finished" podID="ac127502-6058-46d8-a573-49c6a83bd7e7" containerID="2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161" exitCode=0 Jun 08 22:55:34.163251 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.162866 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86c9b6cd6f-2z7pn" Jun 08 22:55:34.163251 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.162894 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9b6cd6f-2z7pn" event={"ID":"ac127502-6058-46d8-a573-49c6a83bd7e7","Type":"ContainerDied","Data":"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161"} Jun 08 22:55:34.163251 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.162956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86c9b6cd6f-2z7pn" event={"ID":"ac127502-6058-46d8-a573-49c6a83bd7e7","Type":"ContainerDied","Data":"63fb9c172fe0ea258b45be213dec72d97d14ad9baf6d1c915ea7f0364034b4a9"} Jun 08 22:55:34.163251 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.162977 2583 scope.go:117] "RemoveContainer" containerID="2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161" Jun 08 22:55:34.172028 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.172006 2583 scope.go:117] "RemoveContainer" containerID="2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161" Jun 08 22:55:34.172286 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:55:34.172269 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161\": container with ID starting with 2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161 not found: ID does not exist" containerID="2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161" Jun 08 22:55:34.172334 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.172294 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161"} err="failed to get container status \"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161\": rpc error: code = NotFound desc = could not find container \"2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161\": container with ID starting with 2ca77a3b9c29e960736a645adbee0e8dad5ebfa1b2d9eb3469eece145ebbd161 not found: ID does not exist" Jun 08 22:55:34.183133 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.183103 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:55:34.186371 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.186343 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86c9b6cd6f-2z7pn"] Jun 08 22:55:34.544031 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:55:34.544012 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac127502-6058-46d8-a573-49c6a83bd7e7" path="/var/lib/kubelet/pods/ac127502-6058-46d8-a573-49c6a83bd7e7/volumes" Jun 08 22:56:13.994404 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:13.994361 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp"] Jun 08 22:56:13.994911 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:13.994663 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac127502-6058-46d8-a573-49c6a83bd7e7" containerName="console" Jun 08 22:56:13.994911 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:13.994675 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac127502-6058-46d8-a573-49c6a83bd7e7" containerName="console" Jun 08 22:56:13.994911 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:13.994744 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac127502-6058-46d8-a573-49c6a83bd7e7" containerName="console" Jun 08 22:56:13.997621 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:13.997603 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.000350 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.000318 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4crw7\"" Jun 08 22:56:14.000516 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.000353 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 08 22:56:14.000581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.000515 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 08 22:56:14.009996 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.009970 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp"] Jun 08 22:56:14.036702 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.036651 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhb59\" (UniqueName: \"kubernetes.io/projected/21f49575-fdf5-480f-b999-03dea5d2b22b-kube-api-access-jhb59\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.036914 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.036718 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21f49575-fdf5-480f-b999-03dea5d2b22b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.137744 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.137701 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhb59\" (UniqueName: \"kubernetes.io/projected/21f49575-fdf5-480f-b999-03dea5d2b22b-kube-api-access-jhb59\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.137744 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.137749 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21f49575-fdf5-480f-b999-03dea5d2b22b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.138113 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.138094 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/21f49575-fdf5-480f-b999-03dea5d2b22b-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.154386 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.154356 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhb59\" (UniqueName: \"kubernetes.io/projected/21f49575-fdf5-480f-b999-03dea5d2b22b-kube-api-access-jhb59\") pod \"cert-manager-operator-controller-manager-57757f7cfc-tfgsp\" (UID: \"21f49575-fdf5-480f-b999-03dea5d2b22b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.306702 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.306675 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" Jun 08 22:56:14.451254 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.451223 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp"] Jun 08 22:56:14.454395 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:56:14.454360 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21f49575_fdf5_480f_b999_03dea5d2b22b.slice/crio-0891822f92c24763f245c8ce8fe6105da793c6686f762b6661dbde178cbea900 WatchSource:0}: Error finding container 0891822f92c24763f245c8ce8fe6105da793c6686f762b6661dbde178cbea900: Status 404 returned error can't find the container with id 0891822f92c24763f245c8ce8fe6105da793c6686f762b6661dbde178cbea900 Jun 08 22:56:14.457890 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:14.457872 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 22:56:15.314491 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:15.314454 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" event={"ID":"21f49575-fdf5-480f-b999-03dea5d2b22b","Type":"ContainerStarted","Data":"0891822f92c24763f245c8ce8fe6105da793c6686f762b6661dbde178cbea900"} Jun 08 22:56:18.330862 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:18.330822 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" event={"ID":"21f49575-fdf5-480f-b999-03dea5d2b22b","Type":"ContainerStarted","Data":"93a64f2c52d7cd8addc51ebefaacc57b9bc9044de47809b984fd47fbd7208a09"} Jun 08 22:56:18.351371 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:18.351315 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-tfgsp" podStartSLOduration=2.136282096 podStartE2EDuration="5.351296515s" totalImagesPullingTime="3.215014419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:56:18.350541993 +0000 UTC m=+500.230449085" watchObservedRunningTime="2026-06-08 22:56:18.351296515 +0000 UTC m=+500.231203607" Jun 08 22:56:20.606060 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.606012 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6"] Jun 08 22:56:20.609549 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.609524 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.612728 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.612701 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 08 22:56:20.612874 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.612736 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 08 22:56:20.613800 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.613785 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-vmqjb\"" Jun 08 22:56:20.619598 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.619574 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6"] Jun 08 22:56:20.695186 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.695147 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgpmr\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-kube-api-access-cgpmr\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.695373 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.695211 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.796284 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.796249 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.796489 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.796305 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgpmr\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-kube-api-access-cgpmr\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.804471 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.804413 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-bound-sa-token\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.804612 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.804482 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgpmr\" (UniqueName: \"kubernetes.io/projected/db8fe998-7935-44c6-afec-e432d2cd41dd-kube-api-access-cgpmr\") pod \"cert-manager-webhook-7dc76cbb94-b7pp6\" (UID: \"db8fe998-7935-44c6-afec-e432d2cd41dd\") " pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:20.926904 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:20.926799 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:21.058499 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:21.058365 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6"] Jun 08 22:56:21.061405 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:56:21.061370 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddb8fe998_7935_44c6_afec_e432d2cd41dd.slice/crio-dd7746ffa48eed16c821a3631c53be1708ff0523670af724b57d0741a85549b6 WatchSource:0}: Error finding container dd7746ffa48eed16c821a3631c53be1708ff0523670af724b57d0741a85549b6: Status 404 returned error can't find the container with id dd7746ffa48eed16c821a3631c53be1708ff0523670af724b57d0741a85549b6 Jun 08 22:56:21.343517 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:21.343478 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" event={"ID":"db8fe998-7935-44c6-afec-e432d2cd41dd","Type":"ContainerStarted","Data":"dd7746ffa48eed16c821a3631c53be1708ff0523670af724b57d0741a85549b6"} Jun 08 22:56:24.358173 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:24.358131 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" event={"ID":"db8fe998-7935-44c6-afec-e432d2cd41dd","Type":"ContainerStarted","Data":"d0de9d1a40fac329982e4e6e4ca78fa32ce7787ba4110ca1cb940e5fc0897551"} Jun 08 22:56:24.358593 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:24.358183 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:24.375820 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:24.375760 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" podStartSLOduration=1.740026675 podStartE2EDuration="4.375743863s" totalImagesPullingTime="2.635717188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:56:24.374145459 +0000 UTC m=+506.254052542" watchObservedRunningTime="2026-06-08 22:56:24.375743863 +0000 UTC m=+506.255650954" Jun 08 22:56:30.364303 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:30.364271 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-7dc76cbb94-b7pp6" Jun 08 22:56:38.938262 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:38.938218 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-59c77684cc-8qh7n"] Jun 08 22:56:38.941612 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:38.941588 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:38.943969 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:38.943944 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-b82zt\"" Jun 08 22:56:38.950159 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:38.950129 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-8qh7n"] Jun 08 22:56:39.053665 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.053632 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s5kq\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-kube-api-access-4s5kq\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.053847 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.053706 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-bound-sa-token\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.154686 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.154643 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-bound-sa-token\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.154856 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.154699 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s5kq\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-kube-api-access-4s5kq\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.162498 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.162468 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-bound-sa-token\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.162630 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.162558 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s5kq\" (UniqueName: \"kubernetes.io/projected/2fbcd88d-e8d0-401c-8e0d-5dca650c58dc-kube-api-access-4s5kq\") pod \"cert-manager-59c77684cc-8qh7n\" (UID: \"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc\") " pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.252035 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.251927 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-59c77684cc-8qh7n" Jun 08 22:56:39.382962 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.382936 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-59c77684cc-8qh7n"] Jun 08 22:56:39.385642 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:56:39.385610 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fbcd88d_e8d0_401c_8e0d_5dca650c58dc.slice/crio-a9f09be2e35daa35770b3aa6cbf742dd4e00286b243dcdf3bcf7e5aeaac05ff5 WatchSource:0}: Error finding container a9f09be2e35daa35770b3aa6cbf742dd4e00286b243dcdf3bcf7e5aeaac05ff5: Status 404 returned error can't find the container with id a9f09be2e35daa35770b3aa6cbf742dd4e00286b243dcdf3bcf7e5aeaac05ff5 Jun 08 22:56:39.422697 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:39.422671 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-8qh7n" event={"ID":"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc","Type":"ContainerStarted","Data":"a9f09be2e35daa35770b3aa6cbf742dd4e00286b243dcdf3bcf7e5aeaac05ff5"} Jun 08 22:56:40.428818 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:40.428779 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-59c77684cc-8qh7n" event={"ID":"2fbcd88d-e8d0-401c-8e0d-5dca650c58dc","Type":"ContainerStarted","Data":"21f8d98f6b0a65057beb494773b74398c5cf566251c7b9d96af9b0e13596f6df"} Jun 08 22:56:40.442984 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:40.442929 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-59c77684cc-8qh7n" podStartSLOduration=2.442912049 podStartE2EDuration="2.442912049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:56:40.44258273 +0000 UTC m=+522.322489823" watchObservedRunningTime="2026-06-08 22:56:40.442912049 +0000 UTC m=+522.322819141" Jun 08 22:56:55.450205 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.450167 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d"] Jun 08 22:56:55.454129 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.454104 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.456727 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.456696 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 08 22:56:55.456876 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.456828 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 08 22:56:55.456876 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.456828 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 08 22:56:55.456991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.456830 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 08 22:56:55.457064 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.457038 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-7hrnz\"" Jun 08 22:56:55.467918 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.467895 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d"] Jun 08 22:56:55.597969 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.597925 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.597969 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.597966 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.598236 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.597995 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/08d5f357-f5f4-490b-9595-e78241eda9e6-kube-api-access-fckrc\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.598236 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.598072 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.598236 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.598146 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699567 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699518 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/08d5f357-f5f4-490b-9595-e78241eda9e6-kube-api-access-fckrc\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699773 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699589 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699773 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699615 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699773 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699689 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699773 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699713 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.699992 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.699973 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-manifests\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.702252 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.702178 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08d5f357-f5f4-490b-9595-e78241eda9e6-tmp\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.702252 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.702219 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-apiservice-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.702409 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.702258 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/08d5f357-f5f4-490b-9595-e78241eda9e6-webhook-cert\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.707689 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.707669 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fckrc\" (UniqueName: \"kubernetes.io/projected/08d5f357-f5f4-490b-9595-e78241eda9e6-kube-api-access-fckrc\") pod \"opendatahub-operator-controller-manager-56cdd97b9c-m888d\" (UID: \"08d5f357-f5f4-490b-9595-e78241eda9e6\") " pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.765717 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.765658 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:56:55.916237 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:55.916196 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d"] Jun 08 22:56:55.920567 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:56:55.920533 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08d5f357_f5f4_490b_9595_e78241eda9e6.slice/crio-a059ea12883c51202c00c0d07753b323428471ecf66fa6e54928e115f6f30781 WatchSource:0}: Error finding container a059ea12883c51202c00c0d07753b323428471ecf66fa6e54928e115f6f30781: Status 404 returned error can't find the container with id a059ea12883c51202c00c0d07753b323428471ecf66fa6e54928e115f6f30781 Jun 08 22:56:56.499070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:56.499028 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" event={"ID":"08d5f357-f5f4-490b-9595-e78241eda9e6","Type":"ContainerStarted","Data":"a059ea12883c51202c00c0d07753b323428471ecf66fa6e54928e115f6f30781"} Jun 08 22:56:59.514561 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:56:59.514521 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" event={"ID":"08d5f357-f5f4-490b-9595-e78241eda9e6","Type":"ContainerStarted","Data":"88131c0e6147e398bbc95426502ccc890f05565fe8c870ad8453d321fc5f582f"} Jun 08 22:57:00.520686 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:00.520648 2583 generic.go:356] "Generic (PLEG): container finished" podID="08d5f357-f5f4-490b-9595-e78241eda9e6" containerID="88131c0e6147e398bbc95426502ccc890f05565fe8c870ad8453d321fc5f582f" exitCode=0 Jun 08 22:57:00.521075 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:00.520735 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" event={"ID":"08d5f357-f5f4-490b-9595-e78241eda9e6","Type":"ContainerDied","Data":"88131c0e6147e398bbc95426502ccc890f05565fe8c870ad8453d321fc5f582f"} Jun 08 22:57:01.527570 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:01.527530 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" event={"ID":"08d5f357-f5f4-490b-9595-e78241eda9e6","Type":"ContainerStarted","Data":"d4c33db2b63c146c7fc4545c5f5d87c7c6486f41011d6292ed04ed991b074832"} Jun 08 22:57:01.528072 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:01.527629 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:57:01.547072 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:01.547005 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" podStartSLOduration=2.463626454 podStartE2EDuration="6.546990425s" totalImagesPullingTime="3.082795382s" totalInitContainerRuntime="1.000568589s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:56:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:01.545705103 +0000 UTC m=+543.425612195" watchObservedRunningTime="2026-06-08 22:57:01.546990425 +0000 UTC m=+543.426897517" Jun 08 22:57:12.533911 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:12.533882 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-56cdd97b9c-m888d" Jun 08 22:57:14.441999 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.441957 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-85448bb448-pscqn"] Jun 08 22:57:14.445814 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.445785 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.449950 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.449925 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 08 22:57:14.450623 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.450594 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 08 22:57:14.460851 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.460823 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85448bb448-pscqn"] Jun 08 22:57:14.573120 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.573086 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c7190-c45f-4bde-954f-6d3238ea6d73-tmp\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.573297 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.573166 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/924c7190-c45f-4bde-954f-6d3238ea6d73-tls-certs\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.573297 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.573185 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ltj4j\" (UniqueName: \"kubernetes.io/projected/924c7190-c45f-4bde-954f-6d3238ea6d73-kube-api-access-ltj4j\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.673648 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.673606 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/924c7190-c45f-4bde-954f-6d3238ea6d73-tls-certs\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.673648 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.673645 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ltj4j\" (UniqueName: \"kubernetes.io/projected/924c7190-c45f-4bde-954f-6d3238ea6d73-kube-api-access-ltj4j\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.673869 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.673691 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c7190-c45f-4bde-954f-6d3238ea6d73-tmp\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.676070 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.676045 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/924c7190-c45f-4bde-954f-6d3238ea6d73-tmp\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.676512 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.676490 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/924c7190-c45f-4bde-954f-6d3238ea6d73-tls-certs\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.687784 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.687758 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ltj4j\" (UniqueName: \"kubernetes.io/projected/924c7190-c45f-4bde-954f-6d3238ea6d73-kube-api-access-ltj4j\") pod \"kube-auth-proxy-85448bb448-pscqn\" (UID: \"924c7190-c45f-4bde-954f-6d3238ea6d73\") " pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.755912 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.755821 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" Jun 08 22:57:14.897457 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:14.897409 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-85448bb448-pscqn"] Jun 08 22:57:14.899576 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:14.899542 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod924c7190_c45f_4bde_954f_6d3238ea6d73.slice/crio-218f73cd5d1d09527e1ecf1aabeddbd86ce9336ad8b719153b8395548a7f4586 WatchSource:0}: Error finding container 218f73cd5d1d09527e1ecf1aabeddbd86ce9336ad8b719153b8395548a7f4586: Status 404 returned error can't find the container with id 218f73cd5d1d09527e1ecf1aabeddbd86ce9336ad8b719153b8395548a7f4586 Jun 08 22:57:15.062564 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.062524 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-72fnh"] Jun 08 22:57:15.066719 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.066697 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.069025 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.069003 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 08 22:57:15.069115 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.069068 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-5hr2v\"" Jun 08 22:57:15.075744 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.075722 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-72fnh"] Jun 08 22:57:15.177865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.177814 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.177865 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.177863 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6jd6\" (UniqueName: \"kubernetes.io/projected/6cff8511-02b0-4726-bac6-668608bacd3e-kube-api-access-p6jd6\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.279011 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.278973 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6jd6\" (UniqueName: \"kubernetes.io/projected/6cff8511-02b0-4726-bac6-668608bacd3e-kube-api-access-p6jd6\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.279248 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.279091 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.279248 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:15.279184 2583 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 08 22:57:15.279248 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:15.279240 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert podName:6cff8511-02b0-4726-bac6-668608bacd3e nodeName:}" failed. No retries permitted until 2026-06-08 22:57:15.779218695 +0000 UTC m=+557.659125783 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert") pod "odh-model-controller-6dc94f689f-72fnh" (UID: "6cff8511-02b0-4726-bac6-668608bacd3e") : secret "odh-model-controller-webhook-cert" not found Jun 08 22:57:15.287423 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.287397 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6jd6\" (UniqueName: \"kubernetes.io/projected/6cff8511-02b0-4726-bac6-668608bacd3e-kube-api-access-p6jd6\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.586495 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.586453 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" event={"ID":"924c7190-c45f-4bde-954f-6d3238ea6d73","Type":"ContainerStarted","Data":"218f73cd5d1d09527e1ecf1aabeddbd86ce9336ad8b719153b8395548a7f4586"} Jun 08 22:57:15.657698 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.657662 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zdhl6"] Jun 08 22:57:15.661294 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.661264 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.664413 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.664384 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-fsjcm\"" Jun 08 22:57:15.664759 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.664735 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 08 22:57:15.686528 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.686370 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zdhl6"] Jun 08 22:57:15.783924 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.783884 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdn6d\" (UniqueName: \"kubernetes.io/projected/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-kube-api-access-kdn6d\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.784126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.784019 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.784126 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.784060 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.788118 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.788044 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6cff8511-02b0-4726-bac6-668608bacd3e-cert\") pod \"odh-model-controller-6dc94f689f-72fnh\" (UID: \"6cff8511-02b0-4726-bac6-668608bacd3e\") " pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:15.884831 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.884729 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdn6d\" (UniqueName: \"kubernetes.io/projected/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-kube-api-access-kdn6d\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.885011 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.884874 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.885011 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:15.884990 2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 08 22:57:15.885116 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:15.885059 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs podName:f4188c8d-e67a-44e7-9a23-e0dbd2f176f5 nodeName:}" failed. No retries permitted until 2026-06-08 22:57:16.385040272 +0000 UTC m=+558.264947359 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs") pod "model-serving-api-5dd488fc48-zdhl6" (UID: "f4188c8d-e67a-44e7-9a23-e0dbd2f176f5") : secret "model-serving-api-tls" not found Jun 08 22:57:15.896775 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.896739 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdn6d\" (UniqueName: \"kubernetes.io/projected/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-kube-api-access-kdn6d\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:15.978336 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:15.978203 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:16.155101 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:16.154911 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-72fnh"] Jun 08 22:57:16.159153 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:16.159122 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cff8511_02b0_4726_bac6_668608bacd3e.slice/crio-9f47cf0208dc6248e04ebf0c3083b07620fe155581bbf6358199511f96f41c8e WatchSource:0}: Error finding container 9f47cf0208dc6248e04ebf0c3083b07620fe155581bbf6358199511f96f41c8e: Status 404 returned error can't find the container with id 9f47cf0208dc6248e04ebf0c3083b07620fe155581bbf6358199511f96f41c8e Jun 08 22:57:16.391422 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:16.391386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:16.391634 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:16.391544 2583 secret.go:189] Couldn't get secret opendatahub/model-serving-api-tls: secret "model-serving-api-tls" not found Jun 08 22:57:16.391634 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:16.391613 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs podName:f4188c8d-e67a-44e7-9a23-e0dbd2f176f5 nodeName:}" failed. No retries permitted until 2026-06-08 22:57:17.391594509 +0000 UTC m=+559.271501583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs") pod "model-serving-api-5dd488fc48-zdhl6" (UID: "f4188c8d-e67a-44e7-9a23-e0dbd2f176f5") : secret "model-serving-api-tls" not found Jun 08 22:57:16.593470 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:16.593402 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" event={"ID":"6cff8511-02b0-4726-bac6-668608bacd3e","Type":"ContainerStarted","Data":"9f47cf0208dc6248e04ebf0c3083b07620fe155581bbf6358199511f96f41c8e"} Jun 08 22:57:17.401088 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:17.400474 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:17.405061 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:17.404983 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f4188c8d-e67a-44e7-9a23-e0dbd2f176f5-tls-certs\") pod \"model-serving-api-5dd488fc48-zdhl6\" (UID: \"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5\") " pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:17.476276 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:17.476235 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:17.924142 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:17.924108 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-zdhl6"] Jun 08 22:57:17.927346 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:17.927302 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4188c8d_e67a_44e7_9a23_e0dbd2f176f5.slice/crio-63f13848166a334784fada1abcbd7472b264f037409fc08b8294a07302088160 WatchSource:0}: Error finding container 63f13848166a334784fada1abcbd7472b264f037409fc08b8294a07302088160: Status 404 returned error can't find the container with id 63f13848166a334784fada1abcbd7472b264f037409fc08b8294a07302088160 Jun 08 22:57:18.606566 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:18.606485 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" event={"ID":"924c7190-c45f-4bde-954f-6d3238ea6d73","Type":"ContainerStarted","Data":"533b61dd080a1abe800854562815cff6daaf5316df0f4b280331668e9c4f56f1"} Jun 08 22:57:18.608216 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:18.608150 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" event={"ID":"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5","Type":"ContainerStarted","Data":"63f13848166a334784fada1abcbd7472b264f037409fc08b8294a07302088160"} Jun 08 22:57:18.623237 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:18.623167 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-85448bb448-pscqn" podStartSLOduration=1.675491821 podStartE2EDuration="4.623147713s" totalImagesPullingTime="2.947655892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:18.620787537 +0000 UTC m=+560.500694640" watchObservedRunningTime="2026-06-08 22:57:18.623147713 +0000 UTC m=+560.503054808" Jun 08 22:57:21.626005 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.625956 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" event={"ID":"6cff8511-02b0-4726-bac6-668608bacd3e","Type":"ContainerStarted","Data":"6df75e1c19bf619d790c045421b568a351170b3c6c78bb23cb2d54f6ca0ca64f"} Jun 08 22:57:21.626523 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.626034 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:21.627651 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.627628 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" event={"ID":"f4188c8d-e67a-44e7-9a23-e0dbd2f176f5","Type":"ContainerStarted","Data":"1cb066263aeaed73a347a4b4033078bfae266a6b932876c7a17c62683967c689"} Jun 08 22:57:21.627767 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.627754 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:21.647212 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.647152 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" podStartSLOduration=2.117872071 podStartE2EDuration="6.647136384s" totalImagesPullingTime="4.529264313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:21.644754277 +0000 UTC m=+563.524661383" watchObservedRunningTime="2026-06-08 22:57:21.647136384 +0000 UTC m=+563.527043505" Jun 08 22:57:21.664513 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:21.664455 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" podStartSLOduration=3.9068798879999997 podStartE2EDuration="6.664422907s" totalImagesPullingTime="2.757543019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:21.663366207 +0000 UTC m=+563.543273305" watchObservedRunningTime="2026-06-08 22:57:21.664422907 +0000 UTC m=+563.544330000" Jun 08 22:57:25.664756 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.664719 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fn4fn"] Jun 08 22:57:25.668596 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.668571 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.671305 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.671278 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 08 22:57:25.671683 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.671658 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2bkg7\"" Jun 08 22:57:25.675305 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.675280 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm"] Jun 08 22:57:25.679043 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.679024 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.682888 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.682869 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-fd2l9\"" Jun 08 22:57:25.683027 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.683010 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 08 22:57:25.685634 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.685613 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fn4fn"] Jun 08 22:57:25.691547 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.691523 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm"] Jun 08 22:57:25.783587 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.783550 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzhrl\" (UniqueName: \"kubernetes.io/projected/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-kube-api-access-wzhrl\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.783813 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.783621 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d22e9d-7cb0-41fb-a977-076c1de379b7-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.783813 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.783646 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrwql\" (UniqueName: \"kubernetes.io/projected/10d22e9d-7cb0-41fb-a977-076c1de379b7-kube-api-access-lrwql\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.783813 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.783716 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.885210 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.885166 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzhrl\" (UniqueName: \"kubernetes.io/projected/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-kube-api-access-wzhrl\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.885392 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.885224 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d22e9d-7cb0-41fb-a977-076c1de379b7-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.885392 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.885243 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrwql\" (UniqueName: \"kubernetes.io/projected/10d22e9d-7cb0-41fb-a977-076c1de379b7-kube-api-access-lrwql\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.885392 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.885260 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.885392 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:25.885382 2583 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 08 22:57:25.885620 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:57:25.885452 2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert podName:6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9 nodeName:}" failed. No retries permitted until 2026-06-08 22:57:26.385420437 +0000 UTC m=+568.265327511 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert") pod "kserve-controller-manager-69fc866d76-fn4fn" (UID: "6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9") : secret "kserve-webhook-server-cert" not found Jun 08 22:57:25.887979 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.887950 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/10d22e9d-7cb0-41fb-a977-076c1de379b7-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.900117 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.900085 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrwql\" (UniqueName: \"kubernetes.io/projected/10d22e9d-7cb0-41fb-a977-076c1de379b7-kube-api-access-lrwql\") pod \"llmisvc-controller-manager-864bfd7bc9-8rjkm\" (UID: \"10d22e9d-7cb0-41fb-a977-076c1de379b7\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:25.903214 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.903182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzhrl\" (UniqueName: \"kubernetes.io/projected/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-kube-api-access-wzhrl\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:25.991709 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:25.991622 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:26.158740 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.158702 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm"] Jun 08 22:57:26.164827 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:26.164793 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod10d22e9d_7cb0_41fb_a977_076c1de379b7.slice/crio-fd02ae6b74d53f17be245d9322145367a89656543e7c21af352e732eb8beba2d WatchSource:0}: Error finding container fd02ae6b74d53f17be245d9322145367a89656543e7c21af352e732eb8beba2d: Status 404 returned error can't find the container with id fd02ae6b74d53f17be245d9322145367a89656543e7c21af352e732eb8beba2d Jun 08 22:57:26.388716 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.388671 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:26.391468 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.391374 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9-cert\") pod \"kserve-controller-manager-69fc866d76-fn4fn\" (UID: \"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:26.554228 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.554190 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq"] Jun 08 22:57:26.558902 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.558877 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.564463 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.564397 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 08 22:57:26.564611 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.564497 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-xjdb8\"" Jun 08 22:57:26.564611 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.564502 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 08 22:57:26.579820 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.579788 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:26.583302 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.583275 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq"] Jun 08 22:57:26.652764 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.652684 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" event={"ID":"10d22e9d-7cb0-41fb-a977-076c1de379b7","Type":"ContainerStarted","Data":"fd02ae6b74d53f17be245d9322145367a89656543e7c21af352e732eb8beba2d"} Jun 08 22:57:26.691122 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.691084 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c11ad4c9-7882-4862-8c8f-d20615ec14c4-operator-config\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.691636 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.691133 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7ppz\" (UniqueName: \"kubernetes.io/projected/c11ad4c9-7882-4862-8c8f-d20615ec14c4-kube-api-access-t7ppz\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.762095 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.762068 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fn4fn"] Jun 08 22:57:26.764616 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:26.764590 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f5c3b3d_28bb_4bab_9bda_ecf7f2ccdde9.slice/crio-5d9a51f6bf4f73f86afda306ccabe4bc715cd0fc3f21648b5a8693b5d6dfb5ae WatchSource:0}: Error finding container 5d9a51f6bf4f73f86afda306ccabe4bc715cd0fc3f21648b5a8693b5d6dfb5ae: Status 404 returned error can't find the container with id 5d9a51f6bf4f73f86afda306ccabe4bc715cd0fc3f21648b5a8693b5d6dfb5ae Jun 08 22:57:26.792578 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.792542 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c11ad4c9-7882-4862-8c8f-d20615ec14c4-operator-config\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.792741 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.792602 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7ppz\" (UniqueName: \"kubernetes.io/projected/c11ad4c9-7882-4862-8c8f-d20615ec14c4-kube-api-access-t7ppz\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.795211 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.795182 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/c11ad4c9-7882-4862-8c8f-d20615ec14c4-operator-config\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.805834 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.805804 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7ppz\" (UniqueName: \"kubernetes.io/projected/c11ad4c9-7882-4862-8c8f-d20615ec14c4-kube-api-access-t7ppz\") pod \"servicemesh-operator3-57f65f65fb-4c8lq\" (UID: \"c11ad4c9-7882-4862-8c8f-d20615ec14c4\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:26.870508 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:26.870473 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:27.235093 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.234960 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq"] Jun 08 22:57:27.257388 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:27.256972 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11ad4c9_7882_4862_8c8f_d20615ec14c4.slice/crio-410874e7a1097367c0bfd8f85e2cdf0ec5bf9f29a91ee80755b1a0076423d9af WatchSource:0}: Error finding container 410874e7a1097367c0bfd8f85e2cdf0ec5bf9f29a91ee80755b1a0076423d9af: Status 404 returned error can't find the container with id 410874e7a1097367c0bfd8f85e2cdf0ec5bf9f29a91ee80755b1a0076423d9af Jun 08 22:57:27.614167 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.614100 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-sp98v"] Jun 08 22:57:27.617937 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.617915 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.621236 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.621202 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 08 22:57:27.621417 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.621392 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-zsvn4\"" Jun 08 22:57:27.633304 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.633277 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-sp98v"] Jun 08 22:57:27.661521 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.661399 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" event={"ID":"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9","Type":"ContainerStarted","Data":"5d9a51f6bf4f73f86afda306ccabe4bc715cd0fc3f21648b5a8693b5d6dfb5ae"} Jun 08 22:57:27.664348 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.664313 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" event={"ID":"c11ad4c9-7882-4862-8c8f-d20615ec14c4","Type":"ContainerStarted","Data":"410874e7a1097367c0bfd8f85e2cdf0ec5bf9f29a91ee80755b1a0076423d9af"} Jun 08 22:57:27.702017 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.701973 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th85p\" (UniqueName: \"kubernetes.io/projected/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-kube-api-access-th85p\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.702522 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.702180 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-redirect-config\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.802802 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.802763 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-redirect-config\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.803007 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.802831 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th85p\" (UniqueName: \"kubernetes.io/projected/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-kube-api-access-th85p\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.803630 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.803605 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-redirect-config\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.817253 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.817224 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-th85p\" (UniqueName: \"kubernetes.io/projected/4ebeb7e1-e1c4-48dd-a51e-97e7261751db-kube-api-access-th85p\") pod \"dashboard-redirect-6976cdc864-sp98v\" (UID: \"4ebeb7e1-e1c4-48dd-a51e-97e7261751db\") " pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:27.931649 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:27.931560 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" Jun 08 22:57:28.171991 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:28.169704 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-6976cdc864-sp98v"] Jun 08 22:57:28.670914 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:28.670870 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" event={"ID":"4ebeb7e1-e1c4-48dd-a51e-97e7261751db","Type":"ContainerStarted","Data":"bfa54834aa3c33c277bdf80de99a6c0bf4ddd3064054175a5331d84617929ee7"} Jun 08 22:57:31.689241 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.689189 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" event={"ID":"6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9","Type":"ContainerStarted","Data":"ac26c0300523178ed6db9094e9e8898cad33a94c64670fab635a72e2fb9cea0d"} Jun 08 22:57:31.689694 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.689268 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:57:31.698034 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.698002 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" event={"ID":"c11ad4c9-7882-4862-8c8f-d20615ec14c4","Type":"ContainerStarted","Data":"16e0b753206e1bbfc0721b5e34f072fbd23accb10af1096ce371dc3a825193a7"} Jun 08 22:57:31.698250 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.698230 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:31.699695 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.699672 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" event={"ID":"10d22e9d-7cb0-41fb-a977-076c1de379b7","Type":"ContainerStarted","Data":"5b9bf528fae1e5c09bea664bbf6703a1ac2108994cc8e4c0495ac6d76fdc9722"} Jun 08 22:57:31.705879 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.705708 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" podStartSLOduration=2.648499832 podStartE2EDuration="6.705679976s" totalImagesPullingTime="4.057180144s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:31.70547702 +0000 UTC m=+573.585384140" watchObservedRunningTime="2026-06-08 22:57:31.705679976 +0000 UTC m=+573.585587061" Jun 08 22:57:31.722103 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.722043 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" podStartSLOduration=2.069675803 podStartE2EDuration="6.722026671s" totalImagesPullingTime="4.652350868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:31.720853877 +0000 UTC m=+573.600761253" watchObservedRunningTime="2026-06-08 22:57:31.722026671 +0000 UTC m=+573.601933788" Jun 08 22:57:31.742988 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:31.742923 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" podStartSLOduration=2.184316638 podStartE2EDuration="5.742905235s" totalImagesPullingTime="3.558588597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:31.741724218 +0000 UTC m=+573.621631312" watchObservedRunningTime="2026-06-08 22:57:31.742905235 +0000 UTC m=+573.622812329" Jun 08 22:57:32.634932 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:32.634900 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-72fnh" Jun 08 22:57:32.638091 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:32.638064 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-zdhl6" Jun 08 22:57:32.707218 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:32.707186 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:57:33.713499 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:33.713381 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" event={"ID":"4ebeb7e1-e1c4-48dd-a51e-97e7261751db","Type":"ContainerStarted","Data":"e96c56965fd4ecc27ef2a722307430dfed792c086c5d58e5a867bcd14df2befe"} Jun 08 22:57:33.733267 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:33.733212 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-6976cdc864-sp98v" podStartSLOduration=1.515477859 podStartE2EDuration="6.733192577s" totalImagesPullingTime="5.217714718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:33.731304874 +0000 UTC m=+575.611211966" watchObservedRunningTime="2026-06-08 22:57:33.733192577 +0000 UTC m=+575.613099704" Jun 08 22:57:37.109548 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.109505 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh"] Jun 08 22:57:37.112892 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.112866 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.115292 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.115269 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-f56g9\"" Jun 08 22:57:37.115476 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.115347 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 08 22:57:37.115476 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.115348 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 08 22:57:37.115666 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.115648 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 08 22:57:37.116457 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.116423 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 08 22:57:37.125013 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.124990 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh"] Jun 08 22:57:37.191338 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191293 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191354 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191408 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191482 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gr2zw\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-kube-api-access-gr2zw\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191528 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191550 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.191581 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.191574 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292419 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292372 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292419 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292425 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gr2zw\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-kube-api-access-gr2zw\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292707 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292474 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292707 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292492 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292707 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292512 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292707 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292575 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.292707 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.292614 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.293261 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.293231 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.295429 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.295401 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.295693 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.295671 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.295760 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.295688 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.296013 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.295995 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.301324 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.301296 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.301529 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.301468 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gr2zw\" (UniqueName: \"kubernetes.io/projected/a0e7cb7a-220e-45ee-9b07-43efbc5c6d37-kube-api-access-gr2zw\") pod \"istiod-openshift-gateway-75c67f8887-r46vh\" (UID: \"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.423509 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.423397 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:37.577697 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.577628 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh"] Jun 08 22:57:37.581305 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:57:37.581269 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e7cb7a_220e_45ee_9b07_43efbc5c6d37.slice/crio-ee3099d7631b2a06eb72d927415be3858dc89eeca045ce06ad97c699bdbdaa2a WatchSource:0}: Error finding container ee3099d7631b2a06eb72d927415be3858dc89eeca045ce06ad97c699bdbdaa2a: Status 404 returned error can't find the container with id ee3099d7631b2a06eb72d927415be3858dc89eeca045ce06ad97c699bdbdaa2a Jun 08 22:57:37.732042 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:37.731949 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" event={"ID":"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37","Type":"ContainerStarted","Data":"ee3099d7631b2a06eb72d927415be3858dc89eeca045ce06ad97c699bdbdaa2a"} Jun 08 22:57:41.130946 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:41.130910 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 08 22:57:41.131200 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:41.130978 2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 08 22:57:41.755088 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:41.755054 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" event={"ID":"a0e7cb7a-220e-45ee-9b07-43efbc5c6d37","Type":"ContainerStarted","Data":"9a4aade474e9848b602228ae06a1c1763f7ed31c5640ebaa8587e939197b978e"} Jun 08 22:57:41.755321 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:41.755199 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:41.775738 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:41.775685 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" podStartSLOduration=1.2291671769999999 podStartE2EDuration="4.775665459s" totalImagesPullingTime="3.546498282s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:57:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:57:41.774114514 +0000 UTC m=+583.654021610" watchObservedRunningTime="2026-06-08 22:57:41.775665459 +0000 UTC m=+583.655572550" Jun 08 22:57:42.708886 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:42.708844 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-4c8lq" Jun 08 22:57:42.765127 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:42.765085 2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-r46vh container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 08 22:57:42.765317 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:42.765149 2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" podUID="a0e7cb7a-220e-45ee-9b07-43efbc5c6d37" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 08 22:57:45.764357 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:45.764327 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-r46vh" Jun 08 22:57:58.524401 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:58.524371 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:57:58.524964 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:57:58.524883 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 22:58:02.712681 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:02.712649 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-fn4fn" Jun 08 22:58:03.716316 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:03.716283 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-8rjkm" Jun 08 22:58:54.414622 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.414579 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf"] Jun 08 22:58:54.419900 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.419877 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:54.423841 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.423809 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-77sp8\"" Jun 08 22:58:54.423985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.423841 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 08 22:58:54.423985 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.423808 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 08 22:58:54.428578 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.428551 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf"] Jun 08 22:58:54.498849 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.498812 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-278tn\" (UniqueName: \"kubernetes.io/projected/3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6-kube-api-access-278tn\") pod \"authorino-operator-75cc8d5cb7-sqfjf\" (UID: \"3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:54.600096 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.600050 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-278tn\" (UniqueName: \"kubernetes.io/projected/3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6-kube-api-access-278tn\") pod \"authorino-operator-75cc8d5cb7-sqfjf\" (UID: \"3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:54.611116 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.611080 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-278tn\" (UniqueName: \"kubernetes.io/projected/3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6-kube-api-access-278tn\") pod \"authorino-operator-75cc8d5cb7-sqfjf\" (UID: \"3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:54.731625 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.731528 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:54.873102 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:54.873076 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf"] Jun 08 22:58:54.875521 ip-10-0-141-133 kubenswrapper[2583]: W0608 22:58:54.875484 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b7eeb82_ef9b_45c5_a9ae_c36fd04503d6.slice/crio-16cbcc0f800fbcdbd4fdc8746b238c229d48d718acb5397694573f870a0c5646 WatchSource:0}: Error finding container 16cbcc0f800fbcdbd4fdc8746b238c229d48d718acb5397694573f870a0c5646: Status 404 returned error can't find the container with id 16cbcc0f800fbcdbd4fdc8746b238c229d48d718acb5397694573f870a0c5646 Jun 08 22:58:55.109760 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:55.109725 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" event={"ID":"3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6","Type":"ContainerStarted","Data":"16cbcc0f800fbcdbd4fdc8746b238c229d48d718acb5397694573f870a0c5646"} Jun 08 22:58:58.125552 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:58.125464 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" event={"ID":"3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6","Type":"ContainerStarted","Data":"cbd1644d8580aa7891841e4d52160ae5891d3beee0be3c3c5e76cbac6b0e8786"} Jun 08 22:58:58.125552 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:58.125536 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:58:58.147162 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:58:58.147094 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" podStartSLOduration=1.9051422329999999 podStartE2EDuration="4.147077404s" totalImagesPullingTime="2.241935171s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 22:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 22:58:58.145091418 +0000 UTC m=+660.024998505" watchObservedRunningTime="2026-06-08 22:58:58.147077404 +0000 UTC m=+660.026984494" Jun 08 22:59:01.905256 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:01.905215 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:59:09.132519 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:09.132485 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-sqfjf" Jun 08 22:59:26.935693 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:26.935625 2583 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-779c9bd5bc-rb2sc" podUID="99124f96-e340-4a4d-9848-91f4c43ec607" containerName="console" containerID="cri-o://b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16" gracePeriod=15 Jun 08 22:59:27.189932 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.189851 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:59:27.271592 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.271550 2583 generic.go:356] "Generic (PLEG): container finished" podID="99124f96-e340-4a4d-9848-91f4c43ec607" containerID="b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16" exitCode=0 Jun 08 22:59:27.271768 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.271601 2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-779c9bd5bc-rb2sc" Jun 08 22:59:27.271768 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.271630 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c9bd5bc-rb2sc" event={"ID":"99124f96-e340-4a4d-9848-91f4c43ec607","Type":"ContainerDied","Data":"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16"} Jun 08 22:59:27.271768 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.271686 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-779c9bd5bc-rb2sc" event={"ID":"99124f96-e340-4a4d-9848-91f4c43ec607","Type":"ContainerDied","Data":"bd7b90f930ed249972247c72bcd06d9bd16fa116b1eff5742164da871c088ffb"} Jun 08 22:59:27.271768 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.271708 2583 scope.go:117] "RemoveContainer" containerID="b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16" Jun 08 22:59:27.280368 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.280346 2583 scope.go:117] "RemoveContainer" containerID="b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16" Jun 08 22:59:27.280724 ip-10-0-141-133 kubenswrapper[2583]: E0608 22:59:27.280699 2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16\": container with ID starting with b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16 not found: ID does not exist" containerID="b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16" Jun 08 22:59:27.280813 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.280733 2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16"} err="failed to get container status \"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16\": rpc error: code = NotFound desc = could not find container \"b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16\": container with ID starting with b341464d8a68de2f9bee5ddc2207194180c88d18f4daec7bb9ec3a87367f0b16 not found: ID does not exist" Jun 08 22:59:27.298601 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298564 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298601 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298607 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298788 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298644 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298788 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298730 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bc4j\" (UniqueName: \"kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298788 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298776 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298927 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298800 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.298927 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.298832 2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca\") pod \"99124f96-e340-4a4d-9848-91f4c43ec607\" (UID: \"99124f96-e340-4a4d-9848-91f4c43ec607\") " Jun 08 22:59:27.299193 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.299043 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:59:27.299327 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.299048 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config" (OuterVolumeSpecName: "console-config") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:59:27.299327 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.299312 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:59:27.299433 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.299345 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca" (OuterVolumeSpecName: "service-ca") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 08 22:59:27.301092 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.301063 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:59:27.301574 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.301550 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 08 22:59:27.301628 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.301569 2583 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j" (OuterVolumeSpecName: "kube-api-access-8bc4j") pod "99124f96-e340-4a4d-9848-91f4c43ec607" (UID: "99124f96-e340-4a4d-9848-91f4c43ec607"). InnerVolumeSpecName "kube-api-access-8bc4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 08 22:59:27.400365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400324 2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-service-ca\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400357 2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-console-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400367 2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-trusted-ca-bundle\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400365 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400376 2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400674 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400386 2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bc4j\" (UniqueName: \"kubernetes.io/projected/99124f96-e340-4a4d-9848-91f4c43ec607-kube-api-access-8bc4j\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400674 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400394 2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/99124f96-e340-4a4d-9848-91f4c43ec607-console-oauth-config\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.400674 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.400402 2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/99124f96-e340-4a4d-9848-91f4c43ec607-oauth-serving-cert\") on node \"ip-10-0-141-133.ec2.internal\" DevicePath \"\"" Jun 08 22:59:27.593753 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.593720 2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:59:27.597520 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:27.597489 2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-779c9bd5bc-rb2sc"] Jun 08 22:59:28.545986 ip-10-0-141-133 kubenswrapper[2583]: I0608 22:59:28.545954 2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99124f96-e340-4a4d-9848-91f4c43ec607" path="/var/lib/kubelet/pods/99124f96-e340-4a4d-9848-91f4c43ec607/volumes" Jun 08 23:00:39.469316 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.469276 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-2m6qq"] Jun 08 23:00:39.469765 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.469671 2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99124f96-e340-4a4d-9848-91f4c43ec607" containerName="console" Jun 08 23:00:39.469765 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.469684 2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="99124f96-e340-4a4d-9848-91f4c43ec607" containerName="console" Jun 08 23:00:39.469765 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.469761 2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="99124f96-e340-4a4d-9848-91f4c43ec607" containerName="console" Jun 08 23:00:39.472813 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.472796 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" Jun 08 23:00:39.475276 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.475256 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 08 23:00:39.475505 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.475492 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-rvbcx\"" Jun 08 23:00:39.475561 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.475532 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 08 23:00:39.481739 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.481712 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-2m6qq"] Jun 08 23:00:39.631080 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.631044 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbpfh\" (UniqueName: \"kubernetes.io/projected/d23ffe02-d2e9-48b2-acd6-ba04fba75c78-kube-api-access-mbpfh\") pod \"keycloak-operator-d67dbbb48-2m6qq\" (UID: \"d23ffe02-d2e9-48b2-acd6-ba04fba75c78\") " pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" Jun 08 23:00:39.732513 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.732386 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbpfh\" (UniqueName: \"kubernetes.io/projected/d23ffe02-d2e9-48b2-acd6-ba04fba75c78-kube-api-access-mbpfh\") pod \"keycloak-operator-d67dbbb48-2m6qq\" (UID: \"d23ffe02-d2e9-48b2-acd6-ba04fba75c78\") " pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" Jun 08 23:00:39.740550 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.740515 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbpfh\" (UniqueName: \"kubernetes.io/projected/d23ffe02-d2e9-48b2-acd6-ba04fba75c78-kube-api-access-mbpfh\") pod \"keycloak-operator-d67dbbb48-2m6qq\" (UID: \"d23ffe02-d2e9-48b2-acd6-ba04fba75c78\") " pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" Jun 08 23:00:39.785273 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.785229 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" Jun 08 23:00:39.922855 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:39.922808 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-2m6qq"] Jun 08 23:00:39.925323 ip-10-0-141-133 kubenswrapper[2583]: W0608 23:00:39.925289 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd23ffe02_d2e9_48b2_acd6_ba04fba75c78.slice/crio-d518a85aa3db4560f8ee533d22a61c97f75c5b002d8551b8ad4a0237b9deb197 WatchSource:0}: Error finding container d518a85aa3db4560f8ee533d22a61c97f75c5b002d8551b8ad4a0237b9deb197: Status 404 returned error can't find the container with id d518a85aa3db4560f8ee533d22a61c97f75c5b002d8551b8ad4a0237b9deb197 Jun 08 23:00:40.629614 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:40.629574 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" event={"ID":"d23ffe02-d2e9-48b2-acd6-ba04fba75c78","Type":"ContainerStarted","Data":"d518a85aa3db4560f8ee533d22a61c97f75c5b002d8551b8ad4a0237b9deb197"} Jun 08 23:00:46.667942 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:46.667889 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" event={"ID":"d23ffe02-d2e9-48b2-acd6-ba04fba75c78","Type":"ContainerStarted","Data":"84fc8861e6cccb6deba51fc5d11b49a2795c7ec1d735309260f5337d98f31e91"} Jun 08 23:00:46.686750 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:00:46.686679 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-2m6qq" podStartSLOduration=1.862730257 podStartE2EDuration="7.686661432s" totalImagesPullingTime="5.823931175s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:00:46.684316312 +0000 UTC m=+768.564223405" watchObservedRunningTime="2026-06-08 23:00:46.686661432 +0000 UTC m=+768.566568523" Jun 08 23:02:05.594399 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.594364 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9sn96"] Jun 08 23:02:05.596735 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.596711 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:05.599360 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.599320 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-895pk\"" Jun 08 23:02:05.599528 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.599413 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jun 08 23:02:05.606695 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.606667 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9sn96"] Jun 08 23:02:05.696145 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.696102 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk5jq\" (UniqueName: \"kubernetes.io/projected/9adaa238-3ac3-49f6-804b-d2382f198d82-kube-api-access-rk5jq\") pod \"payload-processing-7c6c859784-9sn96\" (UID: \"9adaa238-3ac3-49f6-804b-d2382f198d82\") " pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:05.797350 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.797282 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk5jq\" (UniqueName: \"kubernetes.io/projected/9adaa238-3ac3-49f6-804b-d2382f198d82-kube-api-access-rk5jq\") pod \"payload-processing-7c6c859784-9sn96\" (UID: \"9adaa238-3ac3-49f6-804b-d2382f198d82\") " pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:05.808096 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.808054 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk5jq\" (UniqueName: \"kubernetes.io/projected/9adaa238-3ac3-49f6-804b-d2382f198d82-kube-api-access-rk5jq\") pod \"payload-processing-7c6c859784-9sn96\" (UID: \"9adaa238-3ac3-49f6-804b-d2382f198d82\") " pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:05.908282 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:05.908180 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:06.048702 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:06.048669 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-7c6c859784-9sn96"] Jun 08 23:02:06.050783 ip-10-0-141-133 kubenswrapper[2583]: W0608 23:02:06.050754 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9adaa238_3ac3_49f6_804b_d2382f198d82.slice/crio-c80eed3aabc60377ef0155c59392d4a3942c1aabb761cd9c9bfa784c1b65a5a6 WatchSource:0}: Error finding container c80eed3aabc60377ef0155c59392d4a3942c1aabb761cd9c9bfa784c1b65a5a6: Status 404 returned error can't find the container with id c80eed3aabc60377ef0155c59392d4a3942c1aabb761cd9c9bfa784c1b65a5a6 Jun 08 23:02:06.052599 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:06.052582 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 23:02:07.061262 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:07.061187 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"c80eed3aabc60377ef0155c59392d4a3942c1aabb761cd9c9bfa784c1b65a5a6"} Jun 08 23:02:09.074586 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:09.074544 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"2906e75bb969eeda8e225245b8597b3fe93f6158fa79117d5e5dfcf34d350fdd"} Jun 08 23:02:09.075015 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:09.074635 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:09.089903 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:09.089853 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podStartSLOduration=1.579386983 podStartE2EDuration="4.089839517s" totalImagesPullingTime="2.510452534s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:02:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:02:09.088794681 +0000 UTC m=+850.968701789" watchObservedRunningTime="2026-06-08 23:02:09.089839517 +0000 UTC m=+850.969746608" Jun 08 23:02:20.082941 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:20.082905 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:02:58.551740 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:58.551711 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:02:58.553514 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:02:58.553482 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:04:09.654515 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:09.654474 2583 generic.go:356] "Generic (PLEG): container finished" podID="9adaa238-3ac3-49f6-804b-d2382f198d82" containerID="2906e75bb969eeda8e225245b8597b3fe93f6158fa79117d5e5dfcf34d350fdd" exitCode=1 Jun 08 23:04:09.654931 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:09.654548 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerDied","Data":"2906e75bb969eeda8e225245b8597b3fe93f6158fa79117d5e5dfcf34d350fdd"} Jun 08 23:04:09.654931 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:09.654863 2583 scope.go:117] "RemoveContainer" containerID="2906e75bb969eeda8e225245b8597b3fe93f6158fa79117d5e5dfcf34d350fdd" Jun 08 23:04:10.080591 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:10.080553 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:04:10.667420 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:10.667385 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e"} Jun 08 23:04:10.667970 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:10.667509 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:04:21.675610 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:04:21.675573 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:06:10.247983 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:10.247946 2583 generic.go:356] "Generic (PLEG): container finished" podID="9adaa238-3ac3-49f6-804b-d2382f198d82" containerID="e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e" exitCode=1 Jun 08 23:06:10.248427 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:10.248023 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerDied","Data":"e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e"} Jun 08 23:06:10.248427 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:10.248088 2583 scope.go:117] "RemoveContainer" containerID="2906e75bb969eeda8e225245b8597b3fe93f6158fa79117d5e5dfcf34d350fdd" Jun 08 23:06:10.248427 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:10.248370 2583 scope.go:117] "RemoveContainer" containerID="e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e" Jun 08 23:06:10.248653 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:06:10.248633 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:06:11.674048 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:11.674000 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:06:11.674489 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:11.674396 2583 scope.go:117] "RemoveContainer" containerID="e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e" Jun 08 23:06:11.674640 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:06:11.674618 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 10s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:06:25.908587 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:25.908547 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:06:25.908982 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:25.908948 2583 scope.go:117] "RemoveContainer" containerID="e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e" Jun 08 23:06:26.332724 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:26.332691 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc"} Jun 08 23:06:26.332952 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:26.332930 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:06:37.339558 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:06:37.339522 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:07:58.578146 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:07:58.578118 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:07:58.581411 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:07:58.581390 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:08:26.921204 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:26.921166 2583 generic.go:356] "Generic (PLEG): container finished" podID="9adaa238-3ac3-49f6-804b-d2382f198d82" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" exitCode=1 Jun 08 23:08:26.921751 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:26.921241 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerDied","Data":"89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc"} Jun 08 23:08:26.921751 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:26.921319 2583 scope.go:117] "RemoveContainer" containerID="e0fc5cd4b5f0da71f7d6d3df90cec022a8a01b8c1eb0b5edb79827c4a554640e" Jun 08 23:08:26.921751 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:26.921729 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:08:26.924193 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:08:26.923062 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:08:27.338553 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:27.338510 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:08:27.927929 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:27.927904 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:08:27.928324 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:08:27.928101 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:08:41.539800 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:41.539760 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:08:41.541165 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:08:41.540004 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:08:45.908365 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:45.908332 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:08:45.908798 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:45.908731 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:08:45.908923 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:08:45.908903 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:08:56.540215 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:56.540177 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:08:57.070071 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:57.070031 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975"} Jun 08 23:08:57.070285 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:08:57.070267 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:09:08.076972 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:09:08.076941 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:10:57.663620 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:57.663584 2583 generic.go:356] "Generic (PLEG): container finished" podID="9adaa238-3ac3-49f6-804b-d2382f198d82" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" exitCode=1 Jun 08 23:10:57.664089 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:57.663631 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerDied","Data":"1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975"} Jun 08 23:10:57.664089 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:57.663709 2583 scope.go:117] "RemoveContainer" containerID="89344a10448dd55256b0a23761511a3ddeb8918483806eb1fd108f5b4572f5fc" Jun 08 23:10:57.664089 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:57.663997 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:10:57.664238 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:10:57.664218 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:10:58.075573 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:58.075531 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:10:58.670494 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:10:58.670471 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:10:58.670896 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:10:58.670707 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:11:05.908751 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:05.908704 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:11:05.909236 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:05.909099 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:11:05.909328 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:11:05.909306 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:11:17.539401 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:17.539368 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:11:17.539822 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:11:17.539600 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:11:30.540395 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:30.540362 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:11:30.540813 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:11:30.540566 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 40s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:11:41.540347 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:41.540259 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:11:41.883349 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:41.883312 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerStarted","Data":"8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94"} Jun 08 23:11:41.883623 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:41.883605 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:11:52.894627 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:11:52.894586 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:12:58.608101 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:12:58.608038 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:12:58.612325 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:12:58.612304 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:13:42.457999 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:42.457964 2583 generic.go:356] "Generic (PLEG): container finished" podID="9adaa238-3ac3-49f6-804b-d2382f198d82" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" exitCode=1 Jun 08 23:13:42.458536 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:42.458037 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" event={"ID":"9adaa238-3ac3-49f6-804b-d2382f198d82","Type":"ContainerDied","Data":"8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94"} Jun 08 23:13:42.458536 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:42.458102 2583 scope.go:117] "RemoveContainer" containerID="1afa39d5c193474b62c46bf37e09a792ded6b348314aed790032c951f40b4975" Jun 08 23:13:42.458536 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:42.458525 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:13:42.458779 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:13:42.458756 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:13:42.892561 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:42.892515 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:13:43.464885 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:43.464850 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:13:43.465268 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:13:43.465095 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:13:45.908461 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:45.908405 2583 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" Jun 08 23:13:45.908825 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:13:45.908759 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:13:45.908960 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:13:45.908939 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:01.540095 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:01.540062 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:14:01.540505 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:14:01.540237 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:15.307014 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:15.306979 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6976cdc864-sp98v_4ebeb7e1-e1c4-48dd-a51e-97e7261751db/nginx/0.log" Jun 08 23:14:15.520461 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:15.520412 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-fn4fn_6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9/manager/0.log" Jun 08 23:14:15.642087 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:15.642003 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-8rjkm_10d22e9d-7cb0-41fb-a977-076c1de379b7/manager/0.log" Jun 08 23:14:15.868402 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:15.868372 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-zdhl6_f4188c8d-e67a-44e7-9a23-e0dbd2f176f5/server/0.log" Jun 08 23:14:15.984885 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:15.984789 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-72fnh_6cff8511-02b0-4726-bac6-668608bacd3e/manager/0.log" Jun 08 23:14:16.197335 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:16.197309 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-m888d_08d5f357-f5f4-490b-9595-e78241eda9e6/copy-manifests/0.log" Jun 08 23:14:16.203990 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:16.203958 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-m888d_08d5f357-f5f4-490b-9595-e78241eda9e6/manager/0.log" Jun 08 23:14:16.545484 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:16.545430 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:14:16.545852 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:14:16.545672 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:17.674785 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:17.674753 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sqfjf_3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6/manager/0.log" Jun 08 23:14:18.782461 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:18.782418 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-r46vh_a0e7cb7a-220e-45ee-9b07-43efbc5c6d37/discovery/0.log" Jun 08 23:14:18.991473 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:18.991428 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85448bb448-pscqn_924c7190-c45f-4bde-954f-6d3238ea6d73/kube-auth-proxy/0.log" Jun 08 23:14:19.317618 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:19.317588 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:19.423415 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:19.423385 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log" Jun 08 23:14:24.652518 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.652481 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4hdn8/must-gather-d7lrd"] Jun 08 23:14:24.655907 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.655888 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.659325 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.659302 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4hdn8\"/\"openshift-service-ca.crt\"" Jun 08 23:14:24.660357 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.660338 2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-4hdn8\"/\"kube-root-ca.crt\"" Jun 08 23:14:24.660462 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.660338 2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-4hdn8\"/\"default-dockercfg-bfkrn\"" Jun 08 23:14:24.666236 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.666204 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/must-gather-d7lrd"] Jun 08 23:14:24.732543 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.732504 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d10ded25-07e2-4235-8507-6929e49d970b-must-gather-output\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.732717 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.732655 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwwn\" (UniqueName: \"kubernetes.io/projected/d10ded25-07e2-4235-8507-6929e49d970b-kube-api-access-9zwwn\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.833870 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.833818 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwwn\" (UniqueName: \"kubernetes.io/projected/d10ded25-07e2-4235-8507-6929e49d970b-kube-api-access-9zwwn\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.833870 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.833884 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d10ded25-07e2-4235-8507-6929e49d970b-must-gather-output\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.834188 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.834171 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/d10ded25-07e2-4235-8507-6929e49d970b-must-gather-output\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.841600 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.841568 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwwn\" (UniqueName: \"kubernetes.io/projected/d10ded25-07e2-4235-8507-6929e49d970b-kube-api-access-9zwwn\") pod \"must-gather-d7lrd\" (UID: \"d10ded25-07e2-4235-8507-6929e49d970b\") " pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:24.966863 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:24.966775 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" Jun 08 23:14:25.109524 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:25.109415 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/must-gather-d7lrd"] Jun 08 23:14:25.111779 ip-10-0-141-133 kubenswrapper[2583]: W0608 23:14:25.111751 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd10ded25_07e2_4235_8507_6929e49d970b.slice/crio-0e977bfa6c8d232ca35c7d4c1212f2fc12a09d2741c4a910d9811c5e78c68af1 WatchSource:0}: Error finding container 0e977bfa6c8d232ca35c7d4c1212f2fc12a09d2741c4a910d9811c5e78c68af1: Status 404 returned error can't find the container with id 0e977bfa6c8d232ca35c7d4c1212f2fc12a09d2741c4a910d9811c5e78c68af1 Jun 08 23:14:25.113832 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:25.113815 2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 08 23:14:25.673289 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:25.673255 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" event={"ID":"d10ded25-07e2-4235-8507-6929e49d970b","Type":"ContainerStarted","Data":"0e977bfa6c8d232ca35c7d4c1212f2fc12a09d2741c4a910d9811c5e78c68af1"} Jun 08 23:14:26.681591 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:26.681549 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" event={"ID":"d10ded25-07e2-4235-8507-6929e49d970b","Type":"ContainerStarted","Data":"4b285c83e90884314d98dccf665edc49165db053e1fda5670ffcaed88f394a27"} Jun 08 23:14:26.682128 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:26.682097 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" event={"ID":"d10ded25-07e2-4235-8507-6929e49d970b","Type":"ContainerStarted","Data":"844a95abaa64b29cdbb07d04b3c2dc82bd95446a45b9e9cd781b774bb3f1e62d"} Jun 08 23:14:26.698631 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:26.698569 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4hdn8/must-gather-d7lrd" podStartSLOduration=1.809499031 podStartE2EDuration="2.698546359s" totalImagesPullingTime="889.047328ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-08 23:14:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:14:26.696269463 +0000 UTC m=+1588.576176555" watchObservedRunningTime="2026-06-08 23:14:26.698546359 +0000 UTC m=+1588.578453451" Jun 08 23:14:27.626826 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:27.626797 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-x7rkj_e1892e47-8908-4673-b782-e55723903d13/global-pull-secret-syncer/0.log" Jun 08 23:14:27.760935 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:27.760897 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-htb6v_4b52ada8-b170-4037-a664-fdd46e23204c/konnectivity-agent/0.log" Jun 08 23:14:27.797464 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:27.797419 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-133.ec2.internal_b7d005ddae688368c7345a44445a431e/haproxy/0.log" Jun 08 23:14:28.548088 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:28.548064 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:14:28.548321 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:14:28.548300 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:31.231529 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:31.231487 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-4c8lq_c11ad4c9-7882-4862-8c8f-d20615ec14c4/sail-operator/0.log" Jun 08 23:14:32.704201 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:32.704160 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-sqfjf_3b7eeb82-ef9b-45c5-a9ae-c36fd04503d6/manager/0.log" Jun 08 23:14:34.946300 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:34.946203 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-wm68d_b4c6411d-5041-46d9-b4e1-5a03d5528b77/monitoring-plugin/0.log" Jun 08 23:14:34.971260 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:34.971229 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/node-exporter/0.log" Jun 08 23:14:34.991362 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:34.991325 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/kube-rbac-proxy/0.log" Jun 08 23:14:35.010472 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:35.010426 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4f5h9_f439438b-d0a7-4d62-bda9-f200d5cc2017/init-textfile/0.log" Jun 08 23:14:35.346244 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:35.346206 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-qctrf_7e2f567e-4642-4d32-9e78-deffc1e3e3c5/prometheus-operator/0.log" Jun 08 23:14:35.373546 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:35.373514 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7f7d445d84-qctrf_7e2f567e-4642-4d32-9e78-deffc1e3e3c5/kube-rbac-proxy/0.log" Jun 08 23:14:39.626416 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.626376 2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt"] Jun 08 23:14:39.632151 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.632126 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.639239 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.639205 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt"] Jun 08 23:14:39.791055 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.791018 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-podres\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.791299 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.791280 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-lib-modules\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.791410 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.791394 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-sys\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.791651 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.791622 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-proc\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.791893 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.791875 2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb8qw\" (UniqueName: \"kubernetes.io/projected/72e49461-1bfd-4dfd-b844-1b9b94a5b809-kube-api-access-cb8qw\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893035 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.892928 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-lib-modules\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893035 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893000 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-sys\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893035 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893033 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-proc\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893356 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893117 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb8qw\" (UniqueName: \"kubernetes.io/projected/72e49461-1bfd-4dfd-b844-1b9b94a5b809-kube-api-access-cb8qw\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893356 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893225 2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-podres\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893472 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893400 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-podres\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893560 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893542 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-lib-modules\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893627 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893610 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-sys\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.893687 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.893673 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/72e49461-1bfd-4dfd-b844-1b9b94a5b809-proc\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.902478 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.902432 2583 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb8qw\" (UniqueName: \"kubernetes.io/projected/72e49461-1bfd-4dfd-b844-1b9b94a5b809-kube-api-access-cb8qw\") pod \"perf-node-gather-daemonset-p7tgt\" (UID: \"72e49461-1bfd-4dfd-b844-1b9b94a5b809\") " pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:39.950914 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:39.950859 2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:40.128183 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.128134 2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt"] Jun 08 23:14:40.147507 ip-10-0-141-133 kubenswrapper[2583]: W0608 23:14:40.140926 2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod72e49461_1bfd_4dfd_b844_1b9b94a5b809.slice/crio-909f531a124ea3b78340d977c8050919ca0724d0b29b9c63b97e977d401ab62b WatchSource:0}: Error finding container 909f531a124ea3b78340d977c8050919ca0724d0b29b9c63b97e977d401ab62b: Status 404 returned error can't find the container with id 909f531a124ea3b78340d977c8050919ca0724d0b29b9c63b97e977d401ab62b Jun 08 23:14:40.413624 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.413496 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvvh2_411ad465-05c7-4a35-8151-b92c1ac48a1e/dns/0.log" Jun 08 23:14:40.429106 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.429063 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vvvh2_411ad465-05c7-4a35-8151-b92c1ac48a1e/kube-rbac-proxy/0.log" Jun 08 23:14:40.486719 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.486687 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vk4sh_d70d3927-89dd-457d-a77c-2d4c4bdaf48a/dns-node-resolver/0.log" Jun 08 23:14:40.781721 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.781620 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" event={"ID":"72e49461-1bfd-4dfd-b844-1b9b94a5b809","Type":"ContainerStarted","Data":"1e842e0018024738c29a8944d05c04978ba081f07d38b4eba3cdac389828f0d8"} Jun 08 23:14:40.782210 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.781803 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:40.782210 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:40.781864 2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" event={"ID":"72e49461-1bfd-4dfd-b844-1b9b94a5b809","Type":"ContainerStarted","Data":"909f531a124ea3b78340d977c8050919ca0724d0b29b9c63b97e977d401ab62b"} Jun 08 23:14:41.017930 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:41.017895 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c48d6f8c8-vv7p8_12d607d3-5fcc-4357-bd03-1c9697641926/registry/0.log" Jun 08 23:14:41.028914 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:41.028883 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6c48d6f8c8-vv7p8_12d607d3-5fcc-4357-bd03-1c9697641926/registry/1.log" Jun 08 23:14:41.088351 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:41.088320 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lcf6r_7948a206-759b-4031-a1bc-af2b2e94ec55/node-ca/0.log" Jun 08 23:14:42.027067 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.027022 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-r46vh_a0e7cb7a-220e-45ee-9b07-43efbc5c6d37/discovery/0.log" Jun 08 23:14:42.067094 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.067061 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85448bb448-pscqn_924c7190-c45f-4bde-954f-6d3238ea6d73/kube-auth-proxy/0.log" Jun 08 23:14:42.166822 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.166782 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:42.167121 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.167101 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:42.185964 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.185923 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log" Jun 08 23:14:42.751337 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:42.751260 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-94b4v_f91f94b7-610c-4949-8a36-e3cf1f35c9d6/serve-healthcheck-canary/0.log" Jun 08 23:14:43.295514 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.295474 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hmw8f_0df19849-731a-4b98-b3a4-26739a64e434/insights-operator/0.log" Jun 08 23:14:43.295999 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.295921 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-896549976-hmw8f_0df19849-731a-4b98-b3a4-26739a64e434/insights-operator/1.log" Jun 08 23:14:43.312306 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.312281 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rf6j8_cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa/kube-rbac-proxy/0.log" Jun 08 23:14:43.327307 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.327278 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rf6j8_cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa/exporter/0.log" Jun 08 23:14:43.344673 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.344648 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rf6j8_cdc44ca0-0f2a-48a6-afb4-f0367a1d06aa/extractor/0.log" Jun 08 23:14:43.539581 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:43.539547 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:14:43.540556 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:14:43.540071 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:45.541576 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.541547 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-6976cdc864-sp98v_4ebeb7e1-e1c4-48dd-a51e-97e7261751db/nginx/0.log" Jun 08 23:14:45.580689 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.580660 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-fn4fn_6f5c3b3d-28bb-4bab-9bda-ecf7f2ccdde9/manager/0.log" Jun 08 23:14:45.659559 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.659515 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-8rjkm_10d22e9d-7cb0-41fb-a977-076c1de379b7/manager/0.log" Jun 08 23:14:45.705730 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.705699 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-zdhl6_f4188c8d-e67a-44e7-9a23-e0dbd2f176f5/server/0.log" Jun 08 23:14:45.782482 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.782361 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-72fnh_6cff8511-02b0-4726-bac6-668608bacd3e/manager/0.log" Jun 08 23:14:45.842220 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.842189 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-m888d_08d5f357-f5f4-490b-9595-e78241eda9e6/manager/0.log" Jun 08 23:14:45.857743 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:45.857714 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-56cdd97b9c-m888d_08d5f357-f5f4-490b-9595-e78241eda9e6/copy-manifests/0.log" Jun 08 23:14:46.799679 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:46.799653 2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" Jun 08 23:14:46.817864 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:46.817812 2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-4hdn8/perf-node-gather-daemonset-p7tgt" podStartSLOduration=7.817796064 podStartE2EDuration="7.817796064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-08 23:14:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-08 23:14:40.798920618 +0000 UTC m=+1602.678827712" watchObservedRunningTime="2026-06-08 23:14:46.817796064 +0000 UTC m=+1608.697703155" Jun 08 23:14:53.562051 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:53.562019 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-r46vh_a0e7cb7a-220e-45ee-9b07-43efbc5c6d37/discovery/0.log" Jun 08 23:14:53.587206 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:53.587177 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85448bb448-pscqn_924c7190-c45f-4bde-954f-6d3238ea6d73/kube-auth-proxy/0.log" Jun 08 23:14:53.665074 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:53.665035 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:53.665258 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:53.665162 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:53.682727 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:53.682699 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log" Jun 08 23:14:54.225049 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.225015 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/kube-multus-additional-cni-plugins/0.log" Jun 08 23:14:54.241041 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.241011 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/egress-router-binary-copy/0.log" Jun 08 23:14:54.256418 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.256385 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/cni-plugins/0.log" Jun 08 23:14:54.275010 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.274949 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/bond-cni-plugin/0.log" Jun 08 23:14:54.291430 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.291403 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/routeoverride-cni/0.log" Jun 08 23:14:54.308226 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.308191 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/whereabouts-cni-bincopy/0.log" Jun 08 23:14:54.324253 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.324225 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7t5hd_dd770a03-362b-4433-bbe6-50a73e3c683e/whereabouts-cni/0.log" Jun 08 23:14:54.599542 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.599507 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-bj7xh_5c74d00f-6de3-434f-8975-abd5eeda6c8e/kube-multus/0.log" Jun 08 23:14:54.645656 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.645622 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lw8qj_cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c/network-metrics-daemon/0.log" Jun 08 23:14:54.659105 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.659079 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lw8qj_cdf1db79-d3a9-4b48-bc18-e8d3fcaa035c/kube-rbac-proxy/0.log" Jun 08 23:14:54.745543 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.745509 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-r46vh_a0e7cb7a-220e-45ee-9b07-43efbc5c6d37/discovery/0.log" Jun 08 23:14:54.776924 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.776889 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85448bb448-pscqn_924c7190-c45f-4bde-954f-6d3238ea6d73/kube-auth-proxy/0.log" Jun 08 23:14:54.859879 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.859809 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:54.860148 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.860133 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:54.877703 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:54.877675 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log" Jun 08 23:14:56.024296 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.024266 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-controller/0.log" Jun 08 23:14:56.036948 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.036917 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/0.log" Jun 08 23:14:56.037126 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.036977 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovn-acl-logging/1.log" Jun 08 23:14:56.057732 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.057703 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/kube-rbac-proxy-node/0.log" Jun 08 23:14:56.078047 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.078021 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/kube-rbac-proxy-ovn-metrics/0.log" Jun 08 23:14:56.090726 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.090703 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/northd/0.log" Jun 08 23:14:56.105480 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.105431 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/nbdb/0.log" Jun 08 23:14:56.122016 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.121975 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/sbdb/0.log" Jun 08 23:14:56.182142 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.182109 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-rlvhw_ec6d6b7e-bd9c-43ae-a251-e9f911f180d6/ovnkube-controller/0.log" Jun 08 23:14:56.502367 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.502330 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-r46vh_a0e7cb7a-220e-45ee-9b07-43efbc5c6d37/discovery/0.log" Jun 08 23:14:56.527626 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.527586 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-85448bb448-pscqn_924c7190-c45f-4bde-954f-6d3238ea6d73/kube-auth-proxy/0.log" Jun 08 23:14:56.548770 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.548735 2583 scope.go:117] "RemoveContainer" containerID="8c3a22c96be22c2ad3a75b5de12a165d8a5931cfa8caacb961edb6d7a6dbcc94" Jun 08 23:14:56.549313 ip-10-0-141-133 kubenswrapper[2583]: E0608 23:14:56.549277 2583 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"payload-processing\" with CrashLoopBackOff: \"back-off 1m20s restarting failed container=payload-processing pod=payload-processing-7c6c859784-9sn96_openshift-ingress(9adaa238-3ac3-49f6-804b-d2382f198d82)\"" pod="openshift-ingress/payload-processing-7c6c859784-9sn96" podUID="9adaa238-3ac3-49f6-804b-d2382f198d82" Jun 08 23:14:56.606282 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.606183 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:56.606282 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.606235 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-7c6c859784-9sn96_9adaa238-3ac3-49f6-804b-d2382f198d82/payload-processing/4.log" Jun 08 23:14:56.618300 ip-10-0-141-133 kubenswrapper[2583]: I0608 23:14:56.618231 2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-79cdcd997f-qf4wj_5148df69-7982-4755-bbe5-6f131cf83393/router/0.log"