Jun 07 09:05:38.145288 ip-10-0-128-125 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 07 09:05:38.145299 ip-10-0-128-125 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 07 09:05:38.145306 ip-10-0-128-125 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 07 09:05:38.145450 ip-10-0-128-125 systemd[1]: Failed to start Kubernetes Kubelet. Jun 07 09:05:48.258809 ip-10-0-128-125 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 07 09:05:48.258824 ip-10-0-128-125 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 63ad13760d6d456080ffda1808965378 -- Jun 07 09:08:27.889080 ip-10-0-128-125 systemd[1]: Starting Kubernetes Kubelet... Jun 07 09:08:28.255280 ip-10-0-128-125 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:28.255280 ip-10-0-128-125 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 07 09:08:28.255280 ip-10-0-128-125 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:28.255280 ip-10-0-128-125 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 07 09:08:28.255280 ip-10-0-128-125 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 07 09:08:28.256224 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.256110 2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 07 09:08:28.261605 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261584 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.261605 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261601 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.261605 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261605 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.261605 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261609 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.261605 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261613 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261616 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261619 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261622 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261625 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261627 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261630 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261633 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261636 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261638 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261641 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261646 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261650 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261653 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261656 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261659 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261662 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261665 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261667 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.261829 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261670 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261673 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261676 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261678 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261681 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261684 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261686 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261689 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261691 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261694 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261696 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261699 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261702 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261705 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261709 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261714 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261730 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261733 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261736 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.262304 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261739 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261742 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261744 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261747 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261749 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261752 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261755 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261758 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261761 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261764 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261767 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261769 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261772 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261774 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261777 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261781 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261784 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261787 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261790 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.262792 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261793 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261796 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261798 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261817 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261821 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261823 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261826 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261829 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261831 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261836 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261838 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261842 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261845 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261847 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261850 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261852 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261856 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261859 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261862 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261865 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.263267 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261867 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261870 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261874 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261876 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261879 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261881 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261884 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261886 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261889 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261891 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261894 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261897 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261899 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261902 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261904 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261907 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261910 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.261912 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262575 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262581 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262584 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.263743 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262588 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262592 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262595 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262598 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262601 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262604 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262607 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262609 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262612 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262615 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262617 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262620 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262623 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262626 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262629 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262631 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262634 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262637 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262639 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262642 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.264274 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262645 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262647 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262650 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262653 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262655 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262658 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262660 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262664 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262668 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262672 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262677 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262680 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262683 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262686 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262689 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262692 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262695 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262698 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262701 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.264768 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262704 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262706 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262709 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262711 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262714 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262716 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262719 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262721 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262725 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262727 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262730 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262733 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262735 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262738 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262740 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262743 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262745 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262748 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262750 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262753 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.265256 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262756 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262758 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262761 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262763 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262766 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262770 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262772 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262775 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262778 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262781 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262783 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262786 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262788 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262791 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262794 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262797 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262800 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262820 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262824 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.265763 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262827 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262830 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262833 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262836 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262838 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262841 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262844 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262846 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262849 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262853 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262855 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262859 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262862 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262865 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262869 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262873 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262875 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.262879 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263270 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.266333 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263276 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263281 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263285 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263288 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263291 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263294 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263297 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263300 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263303 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263305 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263308 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263311 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263314 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263316 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263319 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263322 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263325 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263328 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263330 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.266798 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263333 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263335 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263338 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263343 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263345 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263348 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263351 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263353 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263356 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263359 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263361 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263366 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263368 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263371 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263374 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263376 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263380 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263382 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263385 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263388 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.267282 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263390 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263393 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263395 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263398 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263401 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263403 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263407 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263411 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263414 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263418 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263421 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263425 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263428 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263431 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263434 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263442 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263445 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263448 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263451 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263454 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.267822 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263456 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263459 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263461 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263465 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263468 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263470 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263473 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263476 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263478 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263481 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263483 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263486 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263488 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263491 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263494 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263496 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263499 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263501 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263504 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263507 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.268318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263509 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263512 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263515 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263518 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263520 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263524 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263527 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263532 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263535 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263538 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263541 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263543 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263546 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263549 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263551 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263556 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263559 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263561 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.263564 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263631 2562 flags.go:64] FLAG: --address="0.0.0.0" Jun 07 09:08:28.268826 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263640 2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263648 2562 flags.go:64] FLAG: --anonymous-auth="true" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263655 2562 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263670 2562 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263675 2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263681 2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263685 2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263688 2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263691 2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263694 2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263697 2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263700 2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263703 2562 flags.go:64] FLAG: --cgroup-root="" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263707 2562 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263710 2562 flags.go:64] FLAG: --client-ca-file="" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263713 2562 flags.go:64] FLAG: --cloud-provider="external" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263716 2562 flags.go:64] FLAG: --cluster-dns="[]" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263721 2562 flags.go:64] FLAG: --cluster-domain="" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263724 2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263728 2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263732 2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263735 2562 flags.go:64] FLAG: --container-log-max-files="5" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263739 2562 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 07 09:08:28.269308 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263741 2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263744 2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263748 2562 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263751 2562 flags.go:64] FLAG: --contention-profiling="false" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263755 2562 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263758 2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263761 2562 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263765 2562 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263769 2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263771 2562 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263774 2562 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263777 2562 flags.go:64] FLAG: --enable-load-reader="false" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263780 2562 flags.go:64] FLAG: --enable-server="true" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263783 2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263790 2562 flags.go:64] FLAG: --event-burst="100" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263793 2562 flags.go:64] FLAG: --event-qps="50" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263797 2562 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263799 2562 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263816 2562 flags.go:64] FLAG: --eviction-hard="" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263822 2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263825 2562 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263828 2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263831 2562 flags.go:64] FLAG: --eviction-soft="" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263834 2562 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263837 2562 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 07 09:08:28.269872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263846 2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263849 2562 flags.go:64] FLAG: --experimental-mounter-path="" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263852 2562 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263856 2562 flags.go:64] FLAG: --fail-swap-on="true" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263859 2562 flags.go:64] FLAG: --feature-gates="" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263862 2562 flags.go:64] FLAG: --file-check-frequency="20s" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263865 2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263868 2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263871 2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263875 2562 flags.go:64] FLAG: --healthz-port="10248" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263877 2562 flags.go:64] FLAG: --help="false" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263882 2562 flags.go:64] FLAG: --hostname-override="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263885 2562 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263888 2562 flags.go:64] FLAG: --http-check-frequency="20s" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263891 2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263894 2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263897 2562 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263900 2562 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263903 2562 flags.go:64] FLAG: --image-service-endpoint="" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263906 2562 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263909 2562 flags.go:64] FLAG: --kube-api-burst="100" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263911 2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263914 2562 flags.go:64] FLAG: --kube-api-qps="50" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263917 2562 flags.go:64] FLAG: --kube-reserved="" Jun 07 09:08:28.270641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263920 2562 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263923 2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263926 2562 flags.go:64] FLAG: --kubelet-cgroups="" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263929 2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263932 2562 flags.go:64] FLAG: --lock-file="" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263935 2562 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263938 2562 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263941 2562 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263946 2562 flags.go:64] FLAG: --log-json-split-stream="false" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263949 2562 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263952 2562 flags.go:64] FLAG: --log-text-split-stream="false" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263956 2562 flags.go:64] FLAG: --logging-format="text" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263959 2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263962 2562 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263965 2562 flags.go:64] FLAG: --manifest-url="" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263967 2562 flags.go:64] FLAG: --manifest-url-header="" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263972 2562 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263975 2562 flags.go:64] FLAG: --max-open-files="1000000" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263979 2562 flags.go:64] FLAG: --max-pods="110" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263983 2562 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263986 2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263989 2562 flags.go:64] FLAG: --memory-manager-policy="None" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263992 2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263995 2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.263998 2562 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 07 09:08:28.271274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264001 2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264008 2562 flags.go:64] FLAG: --node-status-max-images="50" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264011 2562 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264014 2562 flags.go:64] FLAG: --oom-score-adj="-999" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264017 2562 flags.go:64] FLAG: --pod-cidr="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264020 2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:746976bfb3a92b93dacc79c151b98fd4c932eef19c3e2c1516b14eebcac9184d" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264025 2562 flags.go:64] FLAG: --pod-manifest-path="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264028 2562 flags.go:64] FLAG: --pod-max-pids="-1" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264031 2562 flags.go:64] FLAG: --pods-per-core="0" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264033 2562 flags.go:64] FLAG: --port="10250" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264036 2562 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264039 2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-045b98945c8d9f941" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264043 2562 flags.go:64] FLAG: --qos-reserved="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264046 2562 flags.go:64] FLAG: --read-only-port="10255" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264048 2562 flags.go:64] FLAG: --register-node="true" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264051 2562 flags.go:64] FLAG: --register-with-taints="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264055 2562 flags.go:64] FLAG: --registry-burst="10" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264059 2562 flags.go:64] FLAG: --registry-qps="5" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264062 2562 flags.go:64] FLAG: --reserved-cpus="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264065 2562 flags.go:64] FLAG: --reserved-memory="" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264068 2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264071 2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264074 2562 flags.go:64] FLAG: --rotate-certificates="false" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264078 2562 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264081 2562 flags.go:64] FLAG: --runonce="false" Jun 07 09:08:28.271922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264083 2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264086 2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264091 2562 flags.go:64] FLAG: --seccomp-default="false" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264093 2562 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264096 2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264099 2562 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264102 2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264105 2562 flags.go:64] FLAG: --storage-driver-password="root" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264108 2562 flags.go:64] FLAG: --storage-driver-secure="false" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264111 2562 flags.go:64] FLAG: --storage-driver-table="stats" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264114 2562 flags.go:64] FLAG: --storage-driver-user="root" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264116 2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264119 2562 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264122 2562 flags.go:64] FLAG: --system-cgroups="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264125 2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264130 2562 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264133 2562 flags.go:64] FLAG: --tls-cert-file="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264136 2562 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264140 2562 flags.go:64] FLAG: --tls-min-version="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264143 2562 flags.go:64] FLAG: --tls-private-key-file="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264146 2562 flags.go:64] FLAG: --topology-manager-policy="none" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264149 2562 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264152 2562 flags.go:64] FLAG: --topology-manager-scope="container" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264155 2562 flags.go:64] FLAG: --v="2" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264158 2562 flags.go:64] FLAG: --version="false" Jun 07 09:08:28.272528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264163 2562 flags.go:64] FLAG: --vmodule="" Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264167 2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.264170 2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264275 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264279 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264282 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264285 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264288 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264291 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264295 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264297 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264300 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264303 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264305 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264308 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264310 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264313 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264316 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264318 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264321 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.273148 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264324 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264326 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264329 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264332 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264334 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264337 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264340 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264342 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264345 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264347 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264350 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264352 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264356 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264360 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264363 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264365 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264368 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264371 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264373 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264376 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.273640 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264378 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264382 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264385 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264388 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264391 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264393 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264396 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264398 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264401 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264404 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264406 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264411 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264414 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264417 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264421 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264424 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264427 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264430 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264433 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.274162 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264436 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264438 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264441 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264443 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264447 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264450 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264453 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264457 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264461 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264464 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264466 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264469 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264472 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264474 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264479 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264481 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264484 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264487 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264489 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.274622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264492 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264495 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264497 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264500 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264502 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264505 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264508 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264511 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264513 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264516 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264518 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264521 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264524 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264526 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264529 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264531 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264534 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264537 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264541 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.275136 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264543 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264546 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264549 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264551 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.264554 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.265166 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.271969 2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.271987 2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.272006 2562 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.272012 2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272066 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272071 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272075 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272078 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272081 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.275623 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272086 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272090 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272094 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272097 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272100 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272103 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272106 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272108 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272111 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272114 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272116 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272119 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272122 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272125 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272128 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272130 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272133 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272136 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272139 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272141 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.276029 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272144 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272147 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272149 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272152 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272156 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272159 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272161 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272164 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272167 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272169 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272172 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272175 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272178 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272180 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272183 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272186 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272189 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272191 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272194 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272198 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.276503 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272202 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272204 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272207 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272210 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272213 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272215 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272218 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272220 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272223 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272226 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272228 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272232 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272236 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272239 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272242 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272245 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272248 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272251 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272254 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.277004 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272257 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272260 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272262 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272265 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272268 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272270 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272273 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272276 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272278 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272281 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272284 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272286 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272289 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272292 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272294 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272297 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272300 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272303 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272306 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272308 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.277463 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272311 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272314 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272317 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272320 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272322 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272325 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272327 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272330 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272333 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272335 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272338 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272341 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272344 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272347 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272349 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.278011 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.272355 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272461 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272466 2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272468 2562 feature_gate.go:328] unrecognized feature gate: Example2 Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272471 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272474 2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272477 2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272479 2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272482 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272485 2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272488 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272490 2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272493 2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272496 2562 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272499 2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272502 2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272505 2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272509 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272512 2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272514 2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 07 09:08:28.278371 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272517 2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272520 2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272522 2562 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272525 2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272528 2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272531 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272534 2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272537 2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272540 2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272544 2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272547 2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272549 2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272552 2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272554 2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272557 2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272560 2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272562 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272565 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272567 2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272570 2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 07 09:08:28.278865 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272572 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272575 2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272577 2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272580 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272583 2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272586 2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272588 2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272591 2562 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272594 2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272597 2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272599 2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272602 2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272604 2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272607 2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272610 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272612 2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272615 2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272617 2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272620 2562 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272623 2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 07 09:08:28.279359 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272625 2562 feature_gate.go:328] unrecognized feature gate: Example Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272628 2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272631 2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272634 2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272636 2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272639 2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272641 2562 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272644 2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272646 2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272649 2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272652 2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272656 2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272659 2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272662 2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272665 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272668 2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272671 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272674 2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272677 2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 07 09:08:28.279870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272679 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272682 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272684 2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272687 2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272689 2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272692 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272695 2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272697 2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272700 2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272702 2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272705 2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272708 2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272710 2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272714 2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272718 2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272721 2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272724 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272727 2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272730 2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 07 09:08:28.280702 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272733 2562 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:28.272736 2562 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.272741 2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.272858 2562 server.go:962] "Client rotation is on, will bootstrap in background" Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.275736 2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.276979 2562 server.go:1019] "Starting client certificate rotation" Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.277088 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.277128 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 07 09:08:28.281459 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.279242 2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 07 09:08:28.284455 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.284435 2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 07 09:08:28.294945 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.294924 2562 log.go:25] "Validated CRI v1 runtime API" Jun 07 09:08:28.297393 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.297376 2562 log.go:25] "Validated CRI v1 image API" Jun 07 09:08:28.298239 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.298224 2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 07 09:08:28.300288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.300270 2562 fs.go:135] Filesystem UUIDs: map[166b4f14-3971-44af-9bc4-ff4e4aaa45f1:/dev/nvme0n1p4 31e03659-71f3-427c-b0f8-02f856c953d8:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jun 07 09:08:28.300359 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.300287 2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 07 09:08:28.305194 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.305166 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 07 09:08:28.305249 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.305110 2562 manager.go:216] Machine: {Timestamp:2026-06-07 09:08:28.303915049 +0000 UTC m=+0.304920162 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101291 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2352ad1de891f513a73b4cdaf75154 SystemUUID:ec2352ad-1de8-91f5-13a7-3b4cdaf75154 BootID:63ad1376-0d6d-4560-80ff-da1808965378 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:dc:1a:f1:1b:9b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:dc:1a:f1:1b:9b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2e:8a:f1:2a:1c:b3 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 07 09:08:28.305249 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.305235 2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 07 09:08:28.305357 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.305335 2562 manager.go:232] Version: {KernelVersion:5.14.0-570.118.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260602-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 07 09:08:28.308423 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.308400 2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 07 09:08:28.308560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.308426 2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-125.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 07 09:08:28.309071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.309061 2562 topology_manager.go:138] "Creating topology manager with none policy" Jun 07 09:08:28.309107 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.309073 2562 container_manager_linux.go:309] "Creating device plugin manager" Jun 07 09:08:28.309107 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.309081 2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 07 09:08:28.309699 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.309689 2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 07 09:08:28.310535 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.310525 2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 07 09:08:28.313841 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.313829 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 07 09:08:28.314084 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314074 2562 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 07 09:08:28.314116 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314104 2562 kubelet.go:486] "Attempting to sync node with API server" Jun 07 09:08:28.314116 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314114 2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 07 09:08:28.314174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314124 2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 07 09:08:28.314174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314133 2562 kubelet.go:398] "Adding apiserver pod source" Jun 07 09:08:28.314174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314142 2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 07 09:08:28.314516 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314503 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 07 09:08:28.314556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.314529 2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 07 09:08:28.317140 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.317123 2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 07 09:08:28.318431 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.318417 2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 07 09:08:28.318471 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.318447 2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 07 09:08:28.319537 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319523 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319545 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319554 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319563 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319575 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319583 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 07 09:08:28.319598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319595 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 07 09:08:28.319794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319603 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 07 09:08:28.319794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319616 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 07 09:08:28.319794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319625 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 07 09:08:28.319794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319644 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 07 09:08:28.319794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.319653 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 07 09:08:28.320497 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.320487 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 07 09:08:28.320549 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.320500 2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 07 09:08:28.323774 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.323757 2562 server.go:1268] "Started kubelet" Jun 07 09:08:28.324030 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.323950 2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 07 09:08:28.324390 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.324336 2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 07 09:08:28.324476 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.324410 2562 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 07 09:08:28.324711 ip-10-0-128-125 systemd[1]: Started Kubernetes Kubelet. Jun 07 09:08:28.325242 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.324974 2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-125.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 07 09:08:28.325242 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.325040 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-125.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 07 09:08:28.325242 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.325178 2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 07 09:08:28.328178 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.328154 2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 07 09:08:28.329383 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.329363 2562 server.go:310] "Adding debug handlers to kubelet server" Jun 07 09:08:28.330142 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.330116 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fjcdr" Jun 07 09:08:28.330761 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.330742 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fjcdr" Jun 07 09:08:28.331526 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.331506 2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 07 09:08:28.332239 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.331314 2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-125.ec2.internal.18b6c15d1c5f2b70 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-125.ec2.internal,UID:ip-10-0-128-125.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-125.ec2.internal,},FirstTimestamp:2026-06-07 09:08:28.32372824 +0000 UTC m=+0.324733353,LastTimestamp:2026-06-07 09:08:28.32372824 +0000 UTC m=+0.324733353,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-125.ec2.internal,}" Jun 07 09:08:28.333971 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.333958 2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 07 09:08:28.333971 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.333964 2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 07 09:08:28.334078 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.334024 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.334572 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334555 2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 07 09:08:28.334689 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334621 2562 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 07 09:08:28.334786 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334762 2562 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 07 09:08:28.334786 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334693 2562 factory.go:55] Registering systemd factory Jun 07 09:08:28.334786 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334789 2562 factory.go:223] Registration of the systemd container factory successfully Jun 07 09:08:28.334958 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334879 2562 reconstruct.go:97] "Volume reconstruction finished" Jun 07 09:08:28.334958 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334891 2562 reconciler.go:29] "Reconciler: start to sync state" Jun 07 09:08:28.335099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.334999 2562 factory.go:153] Registering CRI-O factory Jun 07 09:08:28.335099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.335013 2562 factory.go:223] Registration of the crio container factory successfully Jun 07 09:08:28.335099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.335095 2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 07 09:08:28.335252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.335121 2562 factory.go:103] Registering Raw factory Jun 07 09:08:28.335252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.335135 2562 manager.go:1196] Started watching for new ooms in manager Jun 07 09:08:28.335903 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.335886 2562 manager.go:349] Starting recovery of all containers Jun 07 09:08:28.343737 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.343710 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:28.346262 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.346248 2562 manager.go:354] Recovery completed Jun 07 09:08:28.346575 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.346557 2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-125.ec2.internal\" not found" node="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.349959 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.349947 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.351831 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.351798 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.351922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.351842 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.351922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.351856 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.352953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.352939 2562 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 07 09:08:28.352953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.352946 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.352953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.352950 2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 07 09:08:28.353141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.352965 2562 state_mem.go:36] "Initialized new in-memory state store" Jun 07 09:08:28.353651 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.353634 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.353729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.353661 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.353729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.353671 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.354098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.354082 2562 policy_none.go:49] "None policy: Start" Jun 07 09:08:28.354098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.354100 2562 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 07 09:08:28.354215 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.354110 2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 07 09:08:28.355233 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.355222 2562 policy_none.go:47] "Start" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400409 2562 manager.go:337] "Starting Device Plugin manager" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.400436 2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400446 2562 server.go:87] "Starting device plugin registration server" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400677 2562 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400689 2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400778 2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400879 2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.400889 2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.401376 2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.401420 2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.403147 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.404304 2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.404335 2562 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.404350 2562 kubelet.go:2448] "Starting kubelet main sync loop" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.404380 2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 07 09:08:28.406828 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.406537 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:28.500849 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.500819 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.501700 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.501687 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.501771 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.501729 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.501771 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.501745 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.501771 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.501767 2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.505058 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.505033 2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal"] Jun 07 09:08:28.505142 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.505090 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.506766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.506718 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.506766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.506749 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.506766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.506761 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.508281 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.508269 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.508403 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.508387 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.508438 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.508421 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.509047 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509029 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.509133 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509065 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.509133 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509081 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.509772 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509758 2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.509867 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.509776 2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-125.ec2.internal\": node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.509927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509891 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.509927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509916 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.510022 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.509931 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.511275 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.511259 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.511355 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.511288 2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 07 09:08:28.511983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.511958 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientMemory" Jun 07 09:08:28.511983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.511983 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasNoDiskPressure" Jun 07 09:08:28.512069 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.511995 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeHasSufficientPID" Jun 07 09:08:28.526145 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.526127 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.535408 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.535387 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-125.ec2.internal\" not found" node="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.536067 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.536050 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.536139 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.536081 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.536139 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.536108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0cb84e7292fcc17afc3acef5821ed4b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-125.ec2.internal\" (UID: \"b0cb84e7292fcc17afc3acef5821ed4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.539618 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.539602 2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-125.ec2.internal\" not found" node="ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.627194 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.627169 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.637248 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637225 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.637368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637253 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.637368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0cb84e7292fcc17afc3acef5821ed4b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-125.ec2.internal\" (UID: \"b0cb84e7292fcc17afc3acef5821ed4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.637368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b0cb84e7292fcc17afc3acef5821ed4b-config\") pod \"kube-apiserver-proxy-ip-10-0-128-125.ec2.internal\" (UID: \"b0cb84e7292fcc17afc3acef5821ed4b\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.637368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637330 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.637368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.637338 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b32c4f6f1087e4db23816b3747b4f3b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal\" (UID: \"b32c4f6f1087e4db23816b3747b4f3b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.727782 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.727750 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.828614 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.828521 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:28.839952 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.839937 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.843705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:28.843558 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:28.929412 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:28.929383 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:29.029938 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:29.029907 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:29.130429 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:29.130364 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:29.175709 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.175685 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:29.231144 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:29.231108 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:29.276624 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.276598 2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 07 09:08:29.277197 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.276735 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 07 09:08:29.277197 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.276735 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 07 09:08:29.277197 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.276762 2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 07 09:08:29.314263 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:29.314233 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb32c4f6f1087e4db23816b3747b4f3b9.slice/crio-0a5d5d360e6e7dd313edb137ae53050abcca5d11a13233c8384645ad74e1706b WatchSource:0}: Error finding container 0a5d5d360e6e7dd313edb137ae53050abcca5d11a13233c8384645ad74e1706b: Status 404 returned error can't find the container with id 0a5d5d360e6e7dd313edb137ae53050abcca5d11a13233c8384645ad74e1706b Jun 07 09:08:29.314622 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:29.314606 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cb84e7292fcc17afc3acef5821ed4b.slice/crio-d3952e56da3ef268e71d56112a9270bb9a23f27227bdf04e1d6cb996763c5eff WatchSource:0}: Error finding container d3952e56da3ef268e71d56112a9270bb9a23f27227bdf04e1d6cb996763c5eff: Status 404 returned error can't find the container with id d3952e56da3ef268e71d56112a9270bb9a23f27227bdf04e1d6cb996763c5eff Jun 07 09:08:29.318485 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.318471 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:08:29.332316 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:29.332290 2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-125.ec2.internal\" not found" Jun 07 09:08:29.335646 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.335616 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-06 09:03:28 +0000 UTC" deadline="2028-02-23 06:53:22.73459092 +0000 UTC" Jun 07 09:08:29.335698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.335648 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15021h44m53.398945736s" Jun 07 09:08:29.375364 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.375345 2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:29.408134 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.408054 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" event={"ID":"b0cb84e7292fcc17afc3acef5821ed4b","Type":"ContainerStarted","Data":"d3952e56da3ef268e71d56112a9270bb9a23f27227bdf04e1d6cb996763c5eff"} Jun 07 09:08:29.409122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.409100 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" event={"ID":"b32c4f6f1087e4db23816b3747b4f3b9","Type":"ContainerStarted","Data":"0a5d5d360e6e7dd313edb137ae53050abcca5d11a13233c8384645ad74e1706b"} Jun 07 09:08:29.435031 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.435018 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" Jun 07 09:08:29.446254 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.446237 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 07 09:08:29.447212 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.447201 2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" Jun 07 09:08:29.455856 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:29.455840 2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 07 09:08:30.240347 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.240302 2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:30.315062 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.315030 2562 apiserver.go:52] "Watching apiserver" Jun 07 09:08:30.326511 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.326482 2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 07 09:08:30.327786 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.327757 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz","openshift-cluster-node-tuning-operator/tuned-7g9ql","openshift-dns/node-resolver-vrbp2","openshift-image-registry/node-ca-j8gj8","openshift-multus/multus-additional-cni-plugins-tfhnf","openshift-network-diagnostics/network-check-target-98nt6","openshift-network-operator/iptables-alerter-fnn8t","openshift-ovn-kubernetes/ovnkube-node-t9kk8","kube-system/global-pull-secret-syncer-kvhzr","kube-system/konnectivity-agent-sfrqf","kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal","openshift-multus/multus-gmbth","openshift-multus/network-metrics-daemon-rkp7x"] Jun 07 09:08:30.329309 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.329284 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.330395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.330376 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.331695 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.331668 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.332512 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.332488 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.332607 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.332495 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-pjz5b\"" Jun 07 09:08:30.332607 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.332581 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.332607 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.332586 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 07 09:08:30.332895 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.332876 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.334034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334011 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 07 09:08:30.334174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334084 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.334335 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334290 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.334409 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334394 2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 07 09:08:30.334552 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334534 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 07 09:08:30.334977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334733 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.334977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334771 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.334977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334860 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wmwzk\"" Jun 07 09:08:30.334977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.334862 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.335229 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.335008 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mcfzd\"" Jun 07 09:08:30.335477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.335459 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:30.335565 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.335540 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:30.335953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.335931 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 07 09:08:30.336122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.336100 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.336122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.336111 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5nrcm\"" Jun 07 09:08:30.336296 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.336227 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.336722 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.336706 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 07 09:08:30.336988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.336968 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.337089 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.337071 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:30.337186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.337165 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 07 09:08:30.337870 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.337766 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-m9x2v\"" Jun 07 09:08:30.338772 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.338715 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.340419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.340403 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.341427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.341360 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 07 09:08:30.341538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.341521 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.341851 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.341832 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.342394 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.342374 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.342466 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.342424 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jsmkt\"" Jun 07 09:08:30.343610 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.343569 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.344540 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.343828 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.344540 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.343855 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:30.344540 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.343959 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-jnx7z\"" Jun 07 09:08:30.344540 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.344229 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.345053 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.345034 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 07 09:08:30.345221 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.345196 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tml4x\"" Jun 07 09:08:30.345288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.345274 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 07 09:08:30.347499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.346234 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.347499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347424 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.347749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347475 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-k8s-cni-cncf-io\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.347749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347628 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-bin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.347749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347671 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-hostroot\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.347749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347721 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91483ade-61fa-4480-8667-b5f93faa14c4-tmp-dir\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.348011 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srwqz\" (UniqueName: \"kubernetes.io/projected/91483ade-61fa-4480-8667-b5f93faa14c4-kube-api-access-srwqz\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.348011 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347871 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-serviceca\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.348011 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.347971 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.348155 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348025 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-multus\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348155 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348084 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-kubelet\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348155 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348134 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-multus-daemon-config\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348298 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348211 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91483ade-61fa-4480-8667-b5f93faa14c4-hosts-file\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.348352 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348305 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mdtv\" (UniqueName: \"kubernetes.io/projected/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-kube-api-access-2mdtv\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.348352 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348339 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-system-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348448 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-cnibin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348448 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348423 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-etc-kubernetes\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348537 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348460 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-host\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.348875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348851 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-os-release\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348893 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-netns\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.348957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348920 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.348957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348948 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d9f2d0-f0fd-411f-a375-bccd7049211f-host-slash\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.348974 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csgsx\" (UniqueName: \"kubernetes.io/projected/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-kube-api-access-csgsx\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349002 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349028 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d9f2d0-f0fd-411f-a375-bccd7049211f-iptables-alerter-script\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349054 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qp6gb\" (UniqueName: \"kubernetes.io/projected/93d9f2d0-f0fd-411f-a375-bccd7049211f-kube-api-access-qp6gb\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349090 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-cni-binary-copy\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.349141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349117 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-multus-certs\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349143 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cnibin\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-os-release\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349241 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-socket-dir-parent\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349267 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-conf-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349289 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7d2dw\" (UniqueName: \"kubernetes.io/projected/64765493-ea44-4117-a576-f301ec7e4df1-kube-api-access-7d2dw\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349333 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.349419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349358 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.349933 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349739 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 07 09:08:30.349933 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.349781 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 07 09:08:30.350200 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.350180 2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 07 09:08:30.351732 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.351712 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 07 09:08:30.351850 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.351712 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 07 09:08:30.351850 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.351712 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7khqb\"" Jun 07 09:08:30.351850 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.351785 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 07 09:08:30.352054 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.352035 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 07 09:08:30.383165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.383145 2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rscbm" Jun 07 09:08:30.393329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.393311 2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rscbm" Jun 07 09:08:30.435530 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.435508 2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 07 09:08:30.449568 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449539 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-os-release\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.449690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449580 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-dbus\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.449690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449606 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.449690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449649 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-systemd-units\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.449690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449661 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-os-release\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.449890 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449759 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.449890 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d9f2d0-f0fd-411f-a375-bccd7049211f-iptables-alerter-script\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.449890 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qp6gb\" (UniqueName: \"kubernetes.io/projected/93d9f2d0-f0fd-411f-a375-bccd7049211f-kube-api-access-qp6gb\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.450029 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.449963 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-cni-binary-copy\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450029 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450008 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cnibin\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450034 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-os-release\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cnibin\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450115 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-netd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysconfig\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450145 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-os-release\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450169 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-host\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450197 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-socket-dir-parent\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450219 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-conf-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450255 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7d2dw\" (UniqueName: \"kubernetes.io/projected/64765493-ea44-4117-a576-f301ec7e4df1-kube-api-access-7d2dw\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450264 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-socket-dir-parent\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450282 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450288 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-conf-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450280 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450324 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-ovn\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450350 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-config\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450374 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-lib-modules\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450391 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-k8s-cni-cncf-io\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-bin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450448 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-bin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-k8s-cni-cncf-io\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-hostroot\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450487 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93d9f2d0-f0fd-411f-a375-bccd7049211f-iptables-alerter-script\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91483ade-61fa-4480-8667-b5f93faa14c4-tmp-dir\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450515 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srwqz\" (UniqueName: \"kubernetes.io/projected/91483ade-61fa-4480-8667-b5f93faa14c4-kube-api-access-srwqz\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450533 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450556 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-hostroot\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450569 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-slash\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450607 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-cni-binary-copy\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450632 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-node-log\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.450929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450659 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-kubelet\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450685 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91483ade-61fa-4480-8667-b5f93faa14c4-hosts-file\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450713 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-var-lib-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450739 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-script-lib\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450767 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-kubelet-config\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450785 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-cni-binary-copy\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450792 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450844 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91483ade-61fa-4480-8667-b5f93faa14c4-tmp-dir\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450861 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-kubernetes\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450875 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-kubelet\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450877 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-systemd\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450938 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-cnibin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450904 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/91483ade-61fa-4480-8667-b5f93faa14c4-hosts-file\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.450965 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-etc-kubernetes\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451011 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-etc-kubernetes\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451013 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-cnibin\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451030 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.451697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451060 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-netns\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451080 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-log-socket\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451087 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451106 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-conf\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451149 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-run\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451173 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-var-lib-kubelet\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-netns\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451255 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-netns\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451354 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-system-cni-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451354 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-socket-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451398 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4hmph\" (UniqueName: \"kubernetes.io/projected/03663312-1b0d-49a8-9239-01130fd2019f-kube-api-access-4hmph\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d9f2d0-f0fd-411f-a375-bccd7049211f-host-slash\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451451 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csgsx\" (UniqueName: \"kubernetes.io/projected/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-kube-api-access-csgsx\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451478 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.452329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451506 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451506 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93d9f2d0-f0fd-411f-a375-bccd7049211f-host-slash\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-etc-tuned\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451564 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-multus-certs\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451589 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-kubelet\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451618 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-etc-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451642 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-env-overrides\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-run-multus-certs\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451667 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-device-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451699 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451780 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c96ff84-adc4-4994-b119-12e61233577b-konnectivity-ca\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451840 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-bin\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451906 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451934 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fkj5\" (UniqueName: \"kubernetes.io/projected/d72371b8-2fcd-44d3-8280-1b3af902f221-kube-api-access-9fkj5\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.451965 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452084 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-serviceca\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452085 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-multus-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.452934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452133 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452141 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c96ff84-adc4-4994-b119-12e61233577b-agent-certs\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452204 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-sys-fs\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452238 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-modprobe-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452252 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-sys\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452287 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-tmp\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452354 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-multus\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452381 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-multus-daemon-config\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452405 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-host-var-lib-cni-multus\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452433 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mdtv\" (UniqueName: \"kubernetes.io/projected/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-kube-api-access-2mdtv\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452499 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-serviceca\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452517 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-systemd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4nxwh\" (UniqueName: \"kubernetes.io/projected/0db1eeab-50d9-455d-800c-76e999b9995f-kube-api-access-4nxwh\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452591 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvv2j\" (UniqueName: \"kubernetes.io/projected/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kube-api-access-zvv2j\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452622 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-system-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452653 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-host\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.453416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452695 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db1eeab-50d9-455d-800c-76e999b9995f-ovn-node-metrics-cert\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.454010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452729 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-registration-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.454010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452734 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-host\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.454010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452757 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/64765493-ea44-4117-a576-f301ec7e4df1-system-cni-dir\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.454010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.452863 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/64765493-ea44-4117-a576-f301ec7e4df1-multus-daemon-config\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.458455 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.458431 2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 07 09:08:30.461109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.461087 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7d2dw\" (UniqueName: \"kubernetes.io/projected/64765493-ea44-4117-a576-f301ec7e4df1-kube-api-access-7d2dw\") pod \"multus-gmbth\" (UID: \"64765493-ea44-4117-a576-f301ec7e4df1\") " pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.464034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.464008 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srwqz\" (UniqueName: \"kubernetes.io/projected/91483ade-61fa-4480-8667-b5f93faa14c4-kube-api-access-srwqz\") pod \"node-resolver-vrbp2\" (UID: \"91483ade-61fa-4480-8667-b5f93faa14c4\") " pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.464162 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.464142 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qp6gb\" (UniqueName: \"kubernetes.io/projected/93d9f2d0-f0fd-411f-a375-bccd7049211f-kube-api-access-qp6gb\") pod \"iptables-alerter-fnn8t\" (UID: \"93d9f2d0-f0fd-411f-a375-bccd7049211f\") " pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.464897 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.464871 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csgsx\" (UniqueName: \"kubernetes.io/projected/875807c0-7e1f-4b5d-b2e9-7a5665aa0e07-kube-api-access-csgsx\") pod \"node-ca-j8gj8\" (UID: \"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07\") " pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.465468 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.465450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mdtv\" (UniqueName: \"kubernetes.io/projected/0e1f3537-4417-495d-ada4-ca6c7a9c3ad1-kube-api-access-2mdtv\") pod \"multus-additional-cni-plugins-tfhnf\" (UID: \"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1\") " pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.465579 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.465562 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:30.465628 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.465584 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:30.465628 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.465598 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:30.465698 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.465674 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:30.965639402 +0000 UTC m=+2.966644506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:30.548049 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.547974 2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:30.553406 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553385 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c96ff84-adc4-4994-b119-12e61233577b-agent-certs\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.553513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553412 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-sys-fs\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.553513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553437 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-modprobe-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.553513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-sys\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.553513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553475 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-tmp\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.553513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553498 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-systemd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553522 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4nxwh\" (UniqueName: \"kubernetes.io/projected/0db1eeab-50d9-455d-800c-76e999b9995f-kube-api-access-4nxwh\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553530 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-sys-fs\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-sys\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvv2j\" (UniqueName: \"kubernetes.io/projected/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kube-api-access-zvv2j\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553577 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-systemd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553592 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553620 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db1eeab-50d9-455d-800c-76e999b9995f-ovn-node-metrics-cert\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553645 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-registration-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553670 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-dbus\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553590 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-modprobe-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553733 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-registration-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.553783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553784 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553796 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-systemd-units\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553826 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-dbus\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553855 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553861 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-systemd-units\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553897 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553921 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-netd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysconfig\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553960 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553974 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-netd\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.553966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-host\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554003 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-host\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554006 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysconfig\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554025 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-ovn\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.553968 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554067 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-config\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554082 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-run-ovn\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554102 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-lib-modules\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.554124 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:31.054104035 +0000 UTC m=+3.055109179 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-lib-modules\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-slash\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554269 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-node-log\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554304 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-var-lib-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554317 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-slash\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-script-lib\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554356 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-kubelet-config\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-node-log\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554396 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-var-lib-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554404 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-kubernetes\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554427 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-systemd\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554454 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-kubernetes\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554483 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-netns\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.554886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554508 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-log-socket\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554519 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554562 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-conf\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554585 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-run\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.554609 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554616 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-var-lib-kubelet\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554618 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-config\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554652 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554669 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-log-socket\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:30.554673 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:31.054655193 +0000 UTC m=+3.055660312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554696 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554711 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-socket-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554716 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-systemd\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554769 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-run\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554783 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/cb26a1c4-76fc-418a-b173-c34ed0071bc5-kubelet-config\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554794 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-socket-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554826 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-run-netns\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.555719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554843 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-var-lib-kubelet\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554853 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4hmph\" (UniqueName: \"kubernetes.io/projected/03663312-1b0d-49a8-9239-01130fd2019f-kube-api-access-4hmph\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554944 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-conf\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.554995 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-etc-selinux\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555041 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-etc-tuned\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555104 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-kubelet\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555138 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-etc-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555143 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03663312-1b0d-49a8-9239-01130fd2019f-etc-sysctl-d\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555175 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-kubelet\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555189 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-env-overrides\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-device-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555221 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-etc-openvswitch\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555257 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c96ff84-adc4-4994-b119-12e61233577b-konnectivity-ca\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555283 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-bin\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555312 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fkj5\" (UniqueName: \"kubernetes.io/projected/d72371b8-2fcd-44d3-8280-1b3af902f221-kube-api-access-9fkj5\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.557367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555471 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-ovnkube-script-lib\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555622 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-device-dir\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555632 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0db1eeab-50d9-455d-800c-76e999b9995f-env-overrides\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.555704 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0db1eeab-50d9-455d-800c-76e999b9995f-host-cni-bin\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.556196 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0db1eeab-50d9-455d-800c-76e999b9995f-ovn-node-metrics-cert\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.556205 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/5c96ff84-adc4-4994-b119-12e61233577b-konnectivity-ca\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.556336 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-tmp\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.557149 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03663312-1b0d-49a8-9239-01130fd2019f-etc-tuned\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.558205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.558072 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/5c96ff84-adc4-4994-b119-12e61233577b-agent-certs\") pod \"konnectivity-agent-sfrqf\" (UID: \"5c96ff84-adc4-4994-b119-12e61233577b\") " pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.563686 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.563659 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4nxwh\" (UniqueName: \"kubernetes.io/projected/0db1eeab-50d9-455d-800c-76e999b9995f-kube-api-access-4nxwh\") pod \"ovnkube-node-t9kk8\" (UID: \"0db1eeab-50d9-455d-800c-76e999b9995f\") " pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:30.563800 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.563696 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvv2j\" (UniqueName: \"kubernetes.io/projected/5d62a8fa-6f64-49d1-b76f-8e728870e4cd-kube-api-access-zvv2j\") pod \"aws-ebs-csi-driver-node-864wz\" (UID: \"5d62a8fa-6f64-49d1-b76f-8e728870e4cd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.564195 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.564172 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4hmph\" (UniqueName: \"kubernetes.io/projected/03663312-1b0d-49a8-9239-01130fd2019f-kube-api-access-4hmph\") pod \"tuned-7g9ql\" (UID: \"03663312-1b0d-49a8-9239-01130fd2019f\") " pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.564729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.564709 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fkj5\" (UniqueName: \"kubernetes.io/projected/d72371b8-2fcd-44d3-8280-1b3af902f221-kube-api-access-9fkj5\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:30.642292 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.642255 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fnn8t" Jun 07 09:08:30.650622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.650598 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmbth" Jun 07 09:08:30.659437 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.659417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vrbp2" Jun 07 09:08:30.664101 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.664078 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-j8gj8" Jun 07 09:08:30.672051 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.672027 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" Jun 07 09:08:30.678068 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.678050 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" Jun 07 09:08:30.685970 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.685948 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" Jun 07 09:08:30.692524 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.692505 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:30.698247 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:30.698228 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:31.058405 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.058368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:31.058560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.058436 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:31.058560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.058481 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:31.058560 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058525 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058558 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058611 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:32.058594913 +0000 UTC m=+4.059600015 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058611 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058633 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058648 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:32.058619423 +0000 UTC m=+4.059624526 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:31.058678 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058649 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:31.058910 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:31.058692 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:32.058672823 +0000 UTC m=+4.059677958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:31.107329 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.107105 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91483ade_61fa_4480_8667_b5f93faa14c4.slice/crio-92f9a11122580978b8f29c94801c2a41d5726510f3d652cacf40d5ff16fafab0 WatchSource:0}: Error finding container 92f9a11122580978b8f29c94801c2a41d5726510f3d652cacf40d5ff16fafab0: Status 404 returned error can't find the container with id 92f9a11122580978b8f29c94801c2a41d5726510f3d652cacf40d5ff16fafab0 Jun 07 09:08:31.107840 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.107817 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64765493_ea44_4117_a576_f301ec7e4df1.slice/crio-c7038ed0de47fbbde76bd8a90d3e727864c6275e93d3b04fb86da7bad52a1fc6 WatchSource:0}: Error finding container c7038ed0de47fbbde76bd8a90d3e727864c6275e93d3b04fb86da7bad52a1fc6: Status 404 returned error can't find the container with id c7038ed0de47fbbde76bd8a90d3e727864c6275e93d3b04fb86da7bad52a1fc6 Jun 07 09:08:31.109625 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.109438 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93d9f2d0_f0fd_411f_a375_bccd7049211f.slice/crio-673919a5d1d3c6e19864160bae8e20079ecbaca561063dc55401638e522400bb WatchSource:0}: Error finding container 673919a5d1d3c6e19864160bae8e20079ecbaca561063dc55401638e522400bb: Status 404 returned error can't find the container with id 673919a5d1d3c6e19864160bae8e20079ecbaca561063dc55401638e522400bb Jun 07 09:08:31.114391 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.114370 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875807c0_7e1f_4b5d_b2e9_7a5665aa0e07.slice/crio-f4b28873e959368476dbaa0d22433d5610df127954d81d774e461a5576108ed4 WatchSource:0}: Error finding container f4b28873e959368476dbaa0d22433d5610df127954d81d774e461a5576108ed4: Status 404 returned error can't find the container with id f4b28873e959368476dbaa0d22433d5610df127954d81d774e461a5576108ed4 Jun 07 09:08:31.115137 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.115114 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03663312_1b0d_49a8_9239_01130fd2019f.slice/crio-81bd8657158fc2b9b2142acc6a4725bb9a5d24a411ddb07ff3f7094cd700302c WatchSource:0}: Error finding container 81bd8657158fc2b9b2142acc6a4725bb9a5d24a411ddb07ff3f7094cd700302c: Status 404 returned error can't find the container with id 81bd8657158fc2b9b2142acc6a4725bb9a5d24a411ddb07ff3f7094cd700302c Jun 07 09:08:31.116558 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.116466 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0db1eeab_50d9_455d_800c_76e999b9995f.slice/crio-93bf8117e93dc7b8519f0489bcae08f5de3eb7d1bdb1ca0212bcaf5e6de73158 WatchSource:0}: Error finding container 93bf8117e93dc7b8519f0489bcae08f5de3eb7d1bdb1ca0212bcaf5e6de73158: Status 404 returned error can't find the container with id 93bf8117e93dc7b8519f0489bcae08f5de3eb7d1bdb1ca0212bcaf5e6de73158 Jun 07 09:08:31.118040 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.117873 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e1f3537_4417_495d_ada4_ca6c7a9c3ad1.slice/crio-9d7ad22535a99ad153f7127c2b7170dfdd29a6600fa28dc05860e5e101f936a7 WatchSource:0}: Error finding container 9d7ad22535a99ad153f7127c2b7170dfdd29a6600fa28dc05860e5e101f936a7: Status 404 returned error can't find the container with id 9d7ad22535a99ad153f7127c2b7170dfdd29a6600fa28dc05860e5e101f936a7 Jun 07 09:08:31.119419 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:08:31.119395 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d62a8fa_6f64_49d1_b76f_8e728870e4cd.slice/crio-42e5b33ceaf35a6b4b10530e6f44d4c7763f301914a261c0252d7db8fe2385c8 WatchSource:0}: Error finding container 42e5b33ceaf35a6b4b10530e6f44d4c7763f301914a261c0252d7db8fe2385c8: Status 404 returned error can't find the container with id 42e5b33ceaf35a6b4b10530e6f44d4c7763f301914a261c0252d7db8fe2385c8 Jun 07 09:08:31.395609 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.395556 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-06 09:03:30 +0000 UTC" deadline="2028-02-17 23:57:14.795258424 +0000 UTC" Jun 07 09:08:31.395609 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.395603 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14894h48m43.3996579s" Jun 07 09:08:31.413119 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.413095 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmbth" event={"ID":"64765493-ea44-4117-a576-f301ec7e4df1","Type":"ContainerStarted","Data":"c7038ed0de47fbbde76bd8a90d3e727864c6275e93d3b04fb86da7bad52a1fc6"} Jun 07 09:08:31.414821 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.414783 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" event={"ID":"b0cb84e7292fcc17afc3acef5821ed4b","Type":"ContainerStarted","Data":"5e41d397721cf74a770c2830565aa4a5d21d99bfdb74bb504a4caeb27b38d363"} Jun 07 09:08:31.415792 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.415773 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" event={"ID":"5d62a8fa-6f64-49d1-b76f-8e728870e4cd","Type":"ContainerStarted","Data":"42e5b33ceaf35a6b4b10530e6f44d4c7763f301914a261c0252d7db8fe2385c8"} Jun 07 09:08:31.416927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.416907 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"93bf8117e93dc7b8519f0489bcae08f5de3eb7d1bdb1ca0212bcaf5e6de73158"} Jun 07 09:08:31.417923 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.417902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerStarted","Data":"9d7ad22535a99ad153f7127c2b7170dfdd29a6600fa28dc05860e5e101f936a7"} Jun 07 09:08:31.419115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.419067 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" event={"ID":"03663312-1b0d-49a8-9239-01130fd2019f","Type":"ContainerStarted","Data":"81bd8657158fc2b9b2142acc6a4725bb9a5d24a411ddb07ff3f7094cd700302c"} Jun 07 09:08:31.420114 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.420096 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j8gj8" event={"ID":"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07","Type":"ContainerStarted","Data":"f4b28873e959368476dbaa0d22433d5610df127954d81d774e461a5576108ed4"} Jun 07 09:08:31.423111 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.423091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fnn8t" event={"ID":"93d9f2d0-f0fd-411f-a375-bccd7049211f","Type":"ContainerStarted","Data":"673919a5d1d3c6e19864160bae8e20079ecbaca561063dc55401638e522400bb"} Jun 07 09:08:31.424473 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.424446 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vrbp2" event={"ID":"91483ade-61fa-4480-8667-b5f93faa14c4","Type":"ContainerStarted","Data":"92f9a11122580978b8f29c94801c2a41d5726510f3d652cacf40d5ff16fafab0"} Jun 07 09:08:31.425880 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.425859 2562 generic.go:356] "Generic (PLEG): container finished" podID="b32c4f6f1087e4db23816b3747b4f3b9" containerID="cf4e843b3fd729a186cb01d6303e057671a4f59b1c161d214c235fcce9bf8766" exitCode=0 Jun 07 09:08:31.425954 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.425930 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" event={"ID":"b32c4f6f1087e4db23816b3747b4f3b9","Type":"ContainerDied","Data":"cf4e843b3fd729a186cb01d6303e057671a4f59b1c161d214c235fcce9bf8766"} Jun 07 09:08:31.427023 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.427005 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sfrqf" event={"ID":"5c96ff84-adc4-4994-b119-12e61233577b","Type":"ContainerStarted","Data":"42d020ceb70c3f63431de75173846f1f565f08dfa0acb3381ef251140d0a7b09"} Jun 07 09:08:31.433278 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.433236 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-125.ec2.internal" podStartSLOduration=2.43322057 podStartE2EDuration="2.43322057s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:31.432648599 +0000 UTC m=+3.433653722" watchObservedRunningTime="2026-06-07 09:08:31.43322057 +0000 UTC m=+3.434225693" Jun 07 09:08:31.931737 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:31.931707 2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 07 09:08:32.071848 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.071800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:32.072031 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.071875 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:32.072031 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.071920 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:32.072132 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072038 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:32.072132 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072099 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:34.072081107 +0000 UTC m=+6.073086211 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:32.072239 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072161 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:32.072239 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072194 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:34.072182407 +0000 UTC m=+6.073187510 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:32.072335 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072265 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:32.072335 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072283 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:32.072335 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072296 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:32.072335 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.072331 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:34.072319915 +0000 UTC m=+6.073325018 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:32.396034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.395943 2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-06 09:03:30 +0000 UTC" deadline="2027-12-12 18:03:41.879677104 +0000 UTC" Jun 07 09:08:32.396034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.395983 2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13280h55m9.483698819s" Jun 07 09:08:32.407952 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.407795 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:32.407952 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.407938 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:32.408604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.408356 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:32.408604 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.408467 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:32.408604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.408542 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:32.408787 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:32.408618 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:32.439941 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:32.439863 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" event={"ID":"b32c4f6f1087e4db23816b3747b4f3b9","Type":"ContainerStarted","Data":"1be7daec898666f2c87b1daa11fd6bc6bc50e9ada7e3add194e40bcfb5f851ed"} Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.092663 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.092740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.092792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.092949 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093012 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:38.092992861 +0000 UTC m=+10.093997966 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093431 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093451 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093464 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093511 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:38.093494595 +0000 UTC m=+10.094499699 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093572 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:34.093631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.093609 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:38.093598089 +0000 UTC m=+10.094603203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:34.405400 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.405316 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:34.405559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.405327 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:34.405559 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.405471 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:34.405559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:34.405337 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:34.405559 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.405527 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:34.405867 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:34.405654 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:36.404641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:36.404594 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:36.404641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:36.404622 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:36.405183 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:36.404600 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:36.405183 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:36.404724 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:36.405183 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:36.404836 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:36.405183 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:36.404907 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:38.127630 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.127586 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.127662 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.127703 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127850 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127868 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127891 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127901 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127858 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127931 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:46.127908527 +0000 UTC m=+18.128913629 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127970 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:46.12795794 +0000 UTC m=+18.128963044 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:38.128098 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.127981 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:08:46.127975459 +0000 UTC m=+18.128980560 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:38.406368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.404935 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:38.406368 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.405193 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:38.406368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.405644 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:38.406368 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.405737 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:38.407060 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:38.406931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:38.407060 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:38.407020 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:40.405162 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:40.405103 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:40.405615 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:40.405230 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:40.405615 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:40.405244 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:40.405615 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:40.405347 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:40.405615 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:40.405379 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:40.405615 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:40.405461 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:42.405674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:42.405636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:42.406170 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:42.405636 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:42.406170 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:42.405759 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:42.406170 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:42.405778 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:42.406170 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:42.405853 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:42.406170 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:42.405950 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:44.408087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:44.408009 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:44.408087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:44.408049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:44.408087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:44.408049 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:44.408532 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:44.408169 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:44.408532 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:44.408304 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:44.408532 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:44.408408 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:46.185196 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.185159 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.185224 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.185261 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185340 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185372 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185377 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185389 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185402 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185428 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:02.185404205 +0000 UTC m=+34.186409330 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185453 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:02.185441606 +0000 UTC m=+34.186446708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:08:46.185622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.185468 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:02.185460975 +0000 UTC m=+34.186466076 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:08:46.404936 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.404905 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:46.405116 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.404905 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:46.405116 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.405019 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:46.405206 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.405148 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:46.405206 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:46.404905 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:46.405295 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:46.405271 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:48.405545 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:48.405508 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:48.406021 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:48.405606 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:48.406376 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:48.406230 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:48.406499 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:48.406443 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:48.406560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:48.406507 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:48.408827 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:48.406693 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:50.405586 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.405401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:50.406497 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.405401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:50.406497 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:50.405699 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:50.406497 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:50.405765 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:50.406497 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.405401 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:50.406497 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:50.405874 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:50.482762 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.482683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sfrqf" event={"ID":"5c96ff84-adc4-4994-b119-12e61233577b","Type":"ContainerStarted","Data":"e7fa710572d96939073d77aae72434c6c3adc948b7eaca0252a215e25ccc6975"} Jun 07 09:08:50.484357 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.484331 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmbth" event={"ID":"64765493-ea44-4117-a576-f301ec7e4df1","Type":"ContainerStarted","Data":"e60f50ccbcede9a552e451b32e5058a35c7bb269a07cadf6bf4368916a45b7f7"} Jun 07 09:08:50.485946 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.485922 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" event={"ID":"5d62a8fa-6f64-49d1-b76f-8e728870e4cd","Type":"ContainerStarted","Data":"b1d81fdffddc466fd67160d0f56b736719e99c72f2372ca1f5c1bc152e978dda"} Jun 07 09:08:50.489012 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.488996 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:08:50.489390 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489368 2562 generic.go:356] "Generic (PLEG): container finished" podID="0db1eeab-50d9-455d-800c-76e999b9995f" containerID="58b108be6ec8c1b79d49bfa11878b1b1bb76e228df6dedfb0f8a281b12191100" exitCode=1 Jun 07 09:08:50.489489 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489435 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"922bb7b28c79f6a8be857b453e69ca456e0bab70056d64a77a794bcaa20730e4"} Jun 07 09:08:50.489548 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489526 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"be95ac6d118cb48a9321d40674a9dc603b4b756f9381e848d405cdc3a82dc89d"} Jun 07 09:08:50.489656 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489581 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"7a11b5c99bb31ac305c608824172c79ef703ab6df9626900291cc3def28534dc"} Jun 07 09:08:50.489656 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489626 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"652bb91b8afb9c5d48f3e6df2d87084f6edf317f00603a24b2155ca2ca141c06"} Jun 07 09:08:50.489760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerDied","Data":"58b108be6ec8c1b79d49bfa11878b1b1bb76e228df6dedfb0f8a281b12191100"} Jun 07 09:08:50.489760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.489735 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"249f913845b263d7ce53722797f475dd6717fdb6454ce58b03d3176c2e63ec85"} Jun 07 09:08:50.491138 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.491110 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="fcff3aaa33c21d36fcee8709430e344eae7d773822b80cd2f73b4f4f5cc63201" exitCode=0 Jun 07 09:08:50.491223 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.491147 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"fcff3aaa33c21d36fcee8709430e344eae7d773822b80cd2f73b4f4f5cc63201"} Jun 07 09:08:50.493005 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.492986 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" event={"ID":"03663312-1b0d-49a8-9239-01130fd2019f","Type":"ContainerStarted","Data":"2594b582ff852a3c2694625ec6d664d14e8fc4ce0caef29e3d5b422d8bccb573"} Jun 07 09:08:50.494581 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.494558 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-j8gj8" event={"ID":"875807c0-7e1f-4b5d-b2e9-7a5665aa0e07","Type":"ContainerStarted","Data":"1d7dca4b1b47963f89f2a54545b3c074b8dd1fce1612f5b7eb135a745137e75f"} Jun 07 09:08:50.496031 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.496008 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vrbp2" event={"ID":"91483ade-61fa-4480-8667-b5f93faa14c4","Type":"ContainerStarted","Data":"84e459660bac03a0676e3d6f787cc62e08c089f94575960671aa195f11951f1e"} Jun 07 09:08:50.498545 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.498506 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sfrqf" podStartSLOduration=8.114165964 podStartE2EDuration="22.498493431s" totalImagesPullingTime="14.384327467s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:50.49764454 +0000 UTC m=+22.498649663" watchObservedRunningTime="2026-06-07 09:08:50.498493431 +0000 UTC m=+22.499498553" Jun 07 09:08:50.498655 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.498627 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-125.ec2.internal" podStartSLOduration=21.498618399 podStartE2EDuration="21.498618399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:32.456184214 +0000 UTC m=+4.457189333" watchObservedRunningTime="2026-06-07 09:08:50.498618399 +0000 UTC m=+22.499623525" Jun 07 09:08:50.515451 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.515389 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-gmbth" podStartSLOduration=3.87701666 podStartE2EDuration="22.515382125s" totalImagesPullingTime="18.638365465s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:50.51503224 +0000 UTC m=+22.516037365" watchObservedRunningTime="2026-06-07 09:08:50.515382125 +0000 UTC m=+22.516387265" Jun 07 09:08:50.529463 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.529435 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-vrbp2" podStartSLOduration=3.964526579 podStartE2EDuration="22.529424572s" totalImagesPullingTime="18.564897993s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:50.529088397 +0000 UTC m=+22.530093521" watchObservedRunningTime="2026-06-07 09:08:50.529424572 +0000 UTC m=+22.530429695" Jun 07 09:08:50.543019 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.542984 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-j8gj8" podStartSLOduration=3.9642767 podStartE2EDuration="22.542974161s" totalImagesPullingTime="18.578697461s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:50.542663769 +0000 UTC m=+22.543668892" watchObservedRunningTime="2026-06-07 09:08:50.542974161 +0000 UTC m=+22.543979283" Jun 07 09:08:50.589705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.589643 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7g9ql" podStartSLOduration=4.011433239 podStartE2EDuration="22.58962661s" totalImagesPullingTime="18.578193371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:50.589345327 +0000 UTC m=+22.590350461" watchObservedRunningTime="2026-06-07 09:08:50.58962661 +0000 UTC m=+22.590631734" Jun 07 09:08:50.636082 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.636033 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:50.693182 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:50.693151 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:51.007362 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.007333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:51.007817 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.007786 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:51.244165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.244143 2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 07 09:08:51.412669 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.412509 2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-07T09:08:51.244161324Z","UUID":"6b3da1f0-7730-485e-ae6c-b028d4d22c50","Handler":null,"Name":"","Endpoint":""} Jun 07 09:08:51.413985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.413962 2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 07 09:08:51.413985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.413991 2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 07 09:08:51.499761 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.499732 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fnn8t" event={"ID":"93d9f2d0-f0fd-411f-a375-bccd7049211f","Type":"ContainerStarted","Data":"80e65a60d0b67bbc2bdd6aba2d146d63ce92f060125e38894f2723c80285ad4e"} Jun 07 09:08:51.501693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.501623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" event={"ID":"5d62a8fa-6f64-49d1-b76f-8e728870e4cd","Type":"ContainerStarted","Data":"68de781e93e619132db2767bb4fe64c3d55e7926503e192c6d2e103c249b73b4"} Jun 07 09:08:51.502249 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.502226 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:51.502422 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.502406 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sfrqf" Jun 07 09:08:51.514752 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.514709 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fnn8t" podStartSLOduration=4.952874066 podStartE2EDuration="23.514695615s" totalImagesPullingTime="18.561821549s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:51.514189364 +0000 UTC m=+23.515194485" watchObservedRunningTime="2026-06-07 09:08:51.514695615 +0000 UTC m=+23.515700738" Jun 07 09:08:51.634855 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.634824 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:51.734680 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:51.734644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:52.405003 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:52.404770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:52.405156 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:52.404770 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:52.405156 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:52.405115 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:52.405274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:52.404886 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:52.405274 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:52.405175 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:52.405379 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:52.405328 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:52.507004 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:52.506973 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" event={"ID":"5d62a8fa-6f64-49d1-b76f-8e728870e4cd","Type":"ContainerStarted","Data":"5871c0456298248578a1a8eed25dbbd0768c53eb4b4b29a27b3226b3a93a2b4e"} Jun 07 09:08:52.524174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:52.524130 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-864wz" podStartSLOduration=3.877461861 podStartE2EDuration="24.524110686s" totalImagesPullingTime="20.646648825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:52.524089877 +0000 UTC m=+24.525095001" watchObservedRunningTime="2026-06-07 09:08:52.524110686 +0000 UTC m=+24.525115810" Jun 07 09:08:53.512143 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:53.512117 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:08:53.512626 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:53.512530 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"dcd38de7aa51bbcdbba6f0f98dfb0be8f8d236bd077733e2a0b897272dfe7fb4"} Jun 07 09:08:53.841427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:53.841341 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:53.891953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:53.891917 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:54.404593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:54.404548 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:54.404593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:54.404568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:54.404846 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:54.404687 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:54.404846 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:54.404720 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:54.404846 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:54.404793 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:54.405018 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:54.404885 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:55.520367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.520213 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:08:55.520942 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.520683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"2c12aae02cd0ccf4f285d74c8ce4865d08985493d77c63048f44da39db995823"} Jun 07 09:08:55.521012 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.520974 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:55.521053 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.521032 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:55.521179 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.521163 2562 scope.go:117] "RemoveContainer" containerID="58b108be6ec8c1b79d49bfa11878b1b1bb76e228df6dedfb0f8a281b12191100" Jun 07 09:08:55.522931 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.522912 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="dcb4b5df1741f83dc0059ce1fe91c57b2a3a9a6583e6f00a6a398896a4dc2e3c" exitCode=0 Jun 07 09:08:55.523032 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.522945 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"dcb4b5df1741f83dc0059ce1fe91c57b2a3a9a6583e6f00a6a398896a4dc2e3c"} Jun 07 09:08:55.536101 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.536083 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:55.552952 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.552931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:55.611948 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:55.611927 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:56.405518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.405485 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:56.405658 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:56.405587 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:56.405658 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.405490 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:56.405658 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.405632 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:56.405782 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:56.405719 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:56.405782 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:56.405777 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:56.528575 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.528547 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:08:56.528933 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.528860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" event={"ID":"0db1eeab-50d9-455d-800c-76e999b9995f","Type":"ContainerStarted","Data":"7515fcfb9967123ae032f7eddb152c1cb5bd5a3e9874d241fdfcbb24cffe3e88"} Jun 07 09:08:56.529193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.529170 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:56.543453 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.543432 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:08:56.571877 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.571832 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" podStartSLOduration=9.955912909 podStartE2EDuration="28.57181894s" totalImagesPullingTime="18.615906031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:08:56.569691564 +0000 UTC m=+28.570696688" watchObservedRunningTime="2026-06-07 09:08:56.57181894 +0000 UTC m=+28.572824054" Jun 07 09:08:56.643316 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.643287 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:56.743736 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:56.743676 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:57.001193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.000976 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kvhzr"] Jun 07 09:08:57.001364 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.001279 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:57.001414 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:57.001390 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:57.001777 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.001761 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkp7x"] Jun 07 09:08:57.001866 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.001855 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:57.001961 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:57.001945 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:57.011619 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.011595 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98nt6"] Jun 07 09:08:57.011735 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.011667 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:57.011800 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:57.011739 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:57.098420 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.098393 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:57.149798 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.149771 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:08:57.533194 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.533156 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="7002d89b8d074c9b1b51b93ef6a0d2a3de59ddcc50fbcb000831fcf7d5dcd1ca" exitCode=0 Jun 07 09:08:57.533533 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:57.533219 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"7002d89b8d074c9b1b51b93ef6a0d2a3de59ddcc50fbcb000831fcf7d5dcd1ca"} Jun 07 09:08:58.405857 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:58.405828 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:08:58.406016 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:58.405931 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:08:58.406016 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:58.405965 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:08:58.406016 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:58.405995 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:08:58.406198 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:58.406038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:08:58.406198 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:08:58.406116 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:08:59.539732 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:59.539700 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="2f9a0054287832bb0c875a1c7ef91b8ab7fcfd1f854f0411ac2dc004cbbacb4c" exitCode=0 Jun 07 09:08:59.540413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:08:59.539747 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"2f9a0054287832bb0c875a1c7ef91b8ab7fcfd1f854f0411ac2dc004cbbacb4c"} Jun 07 09:09:00.405518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:00.405484 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:09:00.405694 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:00.405517 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:00.405694 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:00.405627 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:09:00.405832 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:00.405696 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:09:00.405832 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:00.405748 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:00.405937 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:00.405867 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:09:02.207198 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.207162 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:09:02.207198 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.207204 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.207248 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207342 2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207342 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207357 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207379 2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207390 2562 projected.go:196] Error preparing data for projected volume kube-api-access-tw5vc for pod openshift-network-diagnostics/network-check-target-98nt6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207402 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret podName:cb26a1c4-76fc-418a-b173-c34ed0071bc5 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:34.207387306 +0000 UTC m=+66.208392407 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret") pod "global-pull-secret-syncer-kvhzr" (UID: "cb26a1c4-76fc-418a-b173-c34ed0071bc5") : object "kube-system"/"original-pull-secret" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207419 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:34.207409482 +0000 UTC m=+66.208414586 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 07 09:09:02.207859 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.207436 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc podName:d66e4ab5-f3ad-488d-8a30-c802842cd615 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:34.207426755 +0000 UTC m=+66.208431857 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-tw5vc" (UniqueName: "kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc") pod "network-check-target-98nt6" (UID: "d66e4ab5-f3ad-488d-8a30-c802842cd615") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 07 09:09:02.404893 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.404820 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:09:02.405075 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.404819 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:02.405075 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.404966 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:02.405075 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.404967 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:09:02.405384 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.405098 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-98nt6" podUID="d66e4ab5-f3ad-488d-8a30-c802842cd615" Jun 07 09:09:02.405384 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:02.405265 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-kvhzr" podUID="cb26a1c4-76fc-418a-b173-c34ed0071bc5" Jun 07 09:09:02.852640 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.852614 2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-125.ec2.internal" event="NodeReady" Jun 07 09:09:02.852799 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.852720 2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 07 09:09:02.892852 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.892830 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-9pv2c"] Jun 07 09:09:02.914401 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.914380 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4z8nf"] Jun 07 09:09:02.914532 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.914511 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:02.917285 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.917264 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 07 09:09:02.917426 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.917412 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmvt5\"" Jun 07 09:09:02.917503 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.917489 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 07 09:09:02.936520 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.936498 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9pv2c"] Jun 07 09:09:02.936613 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.936527 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z8nf"] Jun 07 09:09:02.936613 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.936604 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:02.939670 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.939647 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 07 09:09:02.940201 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.939927 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 07 09:09:02.940201 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.939954 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 07 09:09:02.940201 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:02.939994 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-gpvh7\"" Jun 07 09:09:03.002685 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.002654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:03.013905 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.013685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-tmp-dir\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.014006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.013942 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-config-volume\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.014006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.013989 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.014094 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.014005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdmjg\" (UniqueName: \"kubernetes.io/projected/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-kube-api-access-qdmjg\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.049266 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.049238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:03.115346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.115346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115319 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdmjg\" (UniqueName: \"kubernetes.io/projected/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-kube-api-access-qdmjg\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.115496 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-tmp-dir\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.115496 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:03.115405 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:03.115496 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115421 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-config-volume\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.115496 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115448 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b820800-1a37-4938-9cd6-0cb826052e09-cert\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.115496 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:03.115476 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:03.615456967 +0000 UTC m=+35.616462083 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:03.115737 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsgh8\" (UniqueName: \"kubernetes.io/projected/7b820800-1a37-4938-9cd6-0cb826052e09-kube-api-access-nsgh8\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.115788 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.115756 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-tmp-dir\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.116180 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.116162 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-config-volume\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.126754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.126727 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdmjg\" (UniqueName: \"kubernetes.io/projected/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-kube-api-access-qdmjg\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.216862 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.216800 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b820800-1a37-4938-9cd6-0cb826052e09-cert\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.217302 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.216876 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsgh8\" (UniqueName: \"kubernetes.io/projected/7b820800-1a37-4938-9cd6-0cb826052e09-kube-api-access-nsgh8\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.234633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.234603 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7b820800-1a37-4938-9cd6-0cb826052e09-cert\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.234761 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.234728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsgh8\" (UniqueName: \"kubernetes.io/projected/7b820800-1a37-4938-9cd6-0cb826052e09-kube-api-access-nsgh8\") pod \"ingress-canary-4z8nf\" (UID: \"7b820800-1a37-4938-9cd6-0cb826052e09\") " pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.246227 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.246203 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4z8nf" Jun 07 09:09:03.620668 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:03.620631 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:03.620867 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:03.620830 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:03.620946 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:03.620920 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:04.62089345 +0000 UTC m=+36.621898552 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:04.405165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.405113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:04.405631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.405113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:04.405631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.405272 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:09:04.408607 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.408586 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 07 09:09:04.410313 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.410179 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 07 09:09:04.410313 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.410192 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 07 09:09:04.410313 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.410224 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jh2fl\"" Jun 07 09:09:04.410313 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.410245 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99ths\"" Jun 07 09:09:04.410313 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.410287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 07 09:09:04.607783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.607759 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4z8nf"] Jun 07 09:09:04.610870 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:09:04.610846 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b820800_1a37_4938_9cd6_0cb826052e09.slice/crio-b5e82fc2fb7525c11a6759bf1b6ed8f8b9b07c1cefccf8fdb8f74b8b50068a67 WatchSource:0}: Error finding container b5e82fc2fb7525c11a6759bf1b6ed8f8b9b07c1cefccf8fdb8f74b8b50068a67: Status 404 returned error can't find the container with id b5e82fc2fb7525c11a6759bf1b6ed8f8b9b07c1cefccf8fdb8f74b8b50068a67 Jun 07 09:09:04.630376 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:04.630326 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:04.630538 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:04.630516 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:04.630608 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:04.630595 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:06.630574016 +0000 UTC m=+38.631579140 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:05.559247 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:05.559043 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="aff3ce8e3d47b15c68cfdf6cb31fde6d2858ecd57beb24429e705ab402e780ef" exitCode=0 Jun 07 09:09:05.559710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:05.559135 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"aff3ce8e3d47b15c68cfdf6cb31fde6d2858ecd57beb24429e705ab402e780ef"} Jun 07 09:09:05.560954 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:05.560909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z8nf" event={"ID":"7b820800-1a37-4938-9cd6-0cb826052e09","Type":"ContainerStarted","Data":"b5e82fc2fb7525c11a6759bf1b6ed8f8b9b07c1cefccf8fdb8f74b8b50068a67"} Jun 07 09:09:06.567058 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:06.567024 2562 generic.go:356] "Generic (PLEG): container finished" podID="0e1f3537-4417-495d-ada4-ca6c7a9c3ad1" containerID="40aac79006863ff1b674790eb5e9fe1bc648d4bc18d854c2c630527bf98b5184" exitCode=0 Jun 07 09:09:06.567464 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:06.567082 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerDied","Data":"40aac79006863ff1b674790eb5e9fe1bc648d4bc18d854c2c630527bf98b5184"} Jun 07 09:09:06.643766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:06.643727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:06.643926 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:06.643891 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:06.643967 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:06.643956 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:10.643941098 +0000 UTC m=+42.644946198 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:07.027056 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.027021 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:07.572865 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.572825 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" event={"ID":"0e1f3537-4417-495d-ada4-ca6c7a9c3ad1","Type":"ContainerStarted","Data":"fc341305bbd6d1e41ce910773492d03d45ab70bdae4d529798aa37fbfe55774d"} Jun 07 09:09:07.574256 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.574234 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4z8nf" event={"ID":"7b820800-1a37-4938-9cd6-0cb826052e09","Type":"ContainerStarted","Data":"6b1922c4ab46b344d5ff949e099b5abfcfec7a6189c2a1cdb524a49ddc247942"} Jun 07 09:09:07.595849 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.595787 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tfhnf" podStartSLOduration=9.163499844 podStartE2EDuration="39.595774474s" totalImagesPullingTime="30.43227463s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:07.593861623 +0000 UTC m=+39.594866747" watchObservedRunningTime="2026-06-07 09:09:07.595774474 +0000 UTC m=+39.596779596" Jun 07 09:09:07.609145 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.609107 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4z8nf" podStartSLOduration=3.528138045 podStartE2EDuration="5.609094814s" totalImagesPullingTime="2.080956769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:07.608511515 +0000 UTC m=+39.609516641" watchObservedRunningTime="2026-06-07 09:09:07.609094814 +0000 UTC m=+39.610099937" Jun 07 09:09:07.680522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.680487 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:07.731400 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:07.731371 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:08.622571 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:08.622512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:08.669867 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:08.669838 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:10.675739 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:10.675708 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:10.676212 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:10.675862 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:10.676212 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:10.675914 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:18.675899293 +0000 UTC m=+50.676904398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:12.923093 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.923060 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j"] Jun 07 09:09:12.928649 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.928632 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:12.933122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.933077 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-pxzbn\"" Jun 07 09:09:12.933252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.933078 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 07 09:09:12.933252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.933197 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 07 09:09:12.933252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.933152 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 07 09:09:12.933438 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.933078 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 07 09:09:12.934026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.934003 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j"] Jun 07 09:09:12.992026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.992005 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45b898ad-9536-44fd-bbc1-ccd153fb9b09-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:12.992127 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:12.992105 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgpgl\" (UniqueName: \"kubernetes.io/projected/45b898ad-9536-44fd-bbc1-ccd153fb9b09-kube-api-access-qgpgl\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.094630 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.094604 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgpgl\" (UniqueName: \"kubernetes.io/projected/45b898ad-9536-44fd-bbc1-ccd153fb9b09-kube-api-access-qgpgl\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.094723 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.094646 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45b898ad-9536-44fd-bbc1-ccd153fb9b09-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.098181 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.098163 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/45b898ad-9536-44fd-bbc1-ccd153fb9b09-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.103325 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.103307 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgpgl\" (UniqueName: \"kubernetes.io/projected/45b898ad-9536-44fd-bbc1-ccd153fb9b09-kube-api-access-qgpgl\") pod \"managed-serviceaccount-addon-agent-5684bc65d4-vt78j\" (UID: \"45b898ad-9536-44fd-bbc1-ccd153fb9b09\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.247402 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.247342 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" Jun 07 09:09:13.363505 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.363479 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j"] Jun 07 09:09:13.366315 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:09:13.366284 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod45b898ad_9536_44fd_bbc1_ccd153fb9b09.slice/crio-36effa624aa837b4dd2ccd72e8e1e8f176e0a5c89fab0f6eb8cadf5c2cba2be6 WatchSource:0}: Error finding container 36effa624aa837b4dd2ccd72e8e1e8f176e0a5c89fab0f6eb8cadf5c2cba2be6: Status 404 returned error can't find the container with id 36effa624aa837b4dd2ccd72e8e1e8f176e0a5c89fab0f6eb8cadf5c2cba2be6 Jun 07 09:09:13.588966 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:13.588902 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" event={"ID":"45b898ad-9536-44fd-bbc1-ccd153fb9b09","Type":"ContainerStarted","Data":"36effa624aa837b4dd2ccd72e8e1e8f176e0a5c89fab0f6eb8cadf5c2cba2be6"} Jun 07 09:09:16.599110 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:16.599080 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" event={"ID":"45b898ad-9536-44fd-bbc1-ccd153fb9b09","Type":"ContainerStarted","Data":"ff706fc31fdfde0dfcee8d074d465bc3342f07a514500afb46ca6ef174200f8f"} Jun 07 09:09:17.617298 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:17.617251 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" podStartSLOduration=2.499079127 podStartE2EDuration="5.61723523s" totalImagesPullingTime="3.118156103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:09:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:17.61709303 +0000 UTC m=+49.618098155" watchObservedRunningTime="2026-06-07 09:09:17.61723523 +0000 UTC m=+49.618240352" Jun 07 09:09:17.619316 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:17.619294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:17.717732 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:17.717700 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:18.731853 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:18.731788 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:18.732307 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:18.731933 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:18.732307 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:18.732008 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:09:34.731992115 +0000 UTC m=+66.732997216 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:28.545448 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:28.545419 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t9kk8" Jun 07 09:09:28.648875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:28.648841 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:28.749095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:28.749061 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:28.850606 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:28.850539 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:09:34.230741 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.230705 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:34.231193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.230766 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:09:34.231193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.230822 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:34.234213 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.234187 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 07 09:09:34.234297 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.234226 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 07 09:09:34.234297 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.234233 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 07 09:09:34.241762 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:34.241744 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 07 09:09:34.242186 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:34.242152 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:10:38.2417821 +0000 UTC m=+130.242787215 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : secret "metrics-daemon-secret" not found Jun 07 09:09:34.244746 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.244717 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 07 09:09:34.245433 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.244941 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/cb26a1c4-76fc-418a-b173-c34ed0071bc5-original-pull-secret\") pod \"global-pull-secret-syncer-kvhzr\" (UID: \"cb26a1c4-76fc-418a-b173-c34ed0071bc5\") " pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:34.255310 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.255288 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tw5vc\" (UniqueName: \"kubernetes.io/projected/d66e4ab5-f3ad-488d-8a30-c802842cd615-kube-api-access-tw5vc\") pod \"network-check-target-98nt6\" (UID: \"d66e4ab5-f3ad-488d-8a30-c802842cd615\") " pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:34.420395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.420366 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-99ths\"" Jun 07 09:09:34.422744 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.422729 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kvhzr" Jun 07 09:09:34.427370 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.427347 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:34.568109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.568074 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kvhzr"] Jun 07 09:09:34.572011 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:09:34.571983 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb26a1c4_76fc_418a_b173_c34ed0071bc5.slice/crio-d10a9c0955e66b14cca209af453a880140d4fbbdbad3915e3d9c3e5aaba43623 WatchSource:0}: Error finding container d10a9c0955e66b14cca209af453a880140d4fbbdbad3915e3d9c3e5aaba43623: Status 404 returned error can't find the container with id d10a9c0955e66b14cca209af453a880140d4fbbdbad3915e3d9c3e5aaba43623 Jun 07 09:09:34.582842 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.582821 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-98nt6"] Jun 07 09:09:34.585351 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:09:34.585329 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd66e4ab5_f3ad_488d_8a30_c802842cd615.slice/crio-977c6b201c819ee2522c37b5fecbfbad36536ee7100a41375a81251b228f552f WatchSource:0}: Error finding container 977c6b201c819ee2522c37b5fecbfbad36536ee7100a41375a81251b228f552f: Status 404 returned error can't find the container with id 977c6b201c819ee2522c37b5fecbfbad36536ee7100a41375a81251b228f552f Jun 07 09:09:34.640706 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.640683 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kvhzr" event={"ID":"cb26a1c4-76fc-418a-b173-c34ed0071bc5","Type":"ContainerStarted","Data":"d10a9c0955e66b14cca209af453a880140d4fbbdbad3915e3d9c3e5aaba43623"} Jun 07 09:09:34.641730 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.641690 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98nt6" event={"ID":"d66e4ab5-f3ad-488d-8a30-c802842cd615","Type":"ContainerStarted","Data":"977c6b201c819ee2522c37b5fecbfbad36536ee7100a41375a81251b228f552f"} Jun 07 09:09:34.734497 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:34.734472 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:09:34.734631 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:34.734616 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:09:34.734682 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:09:34.734670 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:10:06.734653125 +0000 UTC m=+98.735658236 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:09:39.655642 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:39.655609 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kvhzr" event={"ID":"cb26a1c4-76fc-418a-b173-c34ed0071bc5","Type":"ContainerStarted","Data":"bded01cf2eb10e66b7004a72e9474ce522341398ea777c0c5a8d05de632f17a3"} Jun 07 09:09:39.657120 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:39.657091 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-98nt6" event={"ID":"d66e4ab5-f3ad-488d-8a30-c802842cd615","Type":"ContainerStarted","Data":"69753c8da37ed6e881143c4c95469813ca0747e75e3dae407089435f361280c6"} Jun 07 09:09:39.657238 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:39.657228 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:09:39.672024 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:39.671978 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kvhzr" podStartSLOduration=67.447445196 podStartE2EDuration="1m11.67196613s" totalImagesPullingTime="4.224520934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:39.671398725 +0000 UTC m=+71.672403847" watchObservedRunningTime="2026-06-07 09:09:39.67196613 +0000 UTC m=+71.672971253" Jun 07 09:09:39.686231 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:09:39.686195 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-98nt6" podStartSLOduration=67.479679358 podStartE2EDuration="1m11.686186138s" totalImagesPullingTime="4.20650678s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:09:39.685262156 +0000 UTC m=+71.686267280" watchObservedRunningTime="2026-06-07 09:09:39.686186138 +0000 UTC m=+71.687191261" Jun 07 09:10:06.742252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:10:06.742210 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:10:06.742643 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:10:06.742371 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:10:06.742643 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:10:06.742446 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:10.742428727 +0000 UTC m=+162.743433827 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:10:10.662338 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:10:10.662305 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-98nt6" Jun 07 09:10:38.244370 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:10:38.244332 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:10:38.244872 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:10:38.244483 2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 07 09:10:38.244872 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:10:38.244559 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs podName:d72371b8-2fcd-44d3-8280-1b3af902f221 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:40.244541724 +0000 UTC m=+252.245546829 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs") pod "network-metrics-daemon-rkp7x" (UID: "d72371b8-2fcd-44d3-8280-1b3af902f221") : secret "metrics-daemon-secret" not found Jun 07 09:11:05.923826 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:05.923753 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9pv2c" podUID="50a6ee97-e3e3-489c-b2f8-0f20de1fdd57" Jun 07 09:11:06.868179 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:06.868095 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:11:07.428954 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:07.428905 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-rkp7x" podUID="d72371b8-2fcd-44d3-8280-1b3af902f221" Jun 07 09:11:10.756350 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:10.756316 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:11:10.756797 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:10.756440 2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 07 09:11:10.756797 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:10.756527 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls podName:50a6ee97-e3e3-489c-b2f8-0f20de1fdd57 nodeName:}" failed. No retries permitted until 2026-06-07 09:13:12.756505945 +0000 UTC m=+284.757511063 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls") pod "dns-default-9pv2c" (UID: "50a6ee97-e3e3-489c-b2f8-0f20de1fdd57") : secret "dns-default-metrics-tls" not found Jun 07 09:11:16.891543 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:16.891508 2562 generic.go:356] "Generic (PLEG): container finished" podID="45b898ad-9536-44fd-bbc1-ccd153fb9b09" containerID="ff706fc31fdfde0dfcee8d074d465bc3342f07a514500afb46ca6ef174200f8f" exitCode=255 Jun 07 09:11:16.891922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:16.891582 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" event={"ID":"45b898ad-9536-44fd-bbc1-ccd153fb9b09","Type":"ContainerDied","Data":"ff706fc31fdfde0dfcee8d074d465bc3342f07a514500afb46ca6ef174200f8f"} Jun 07 09:11:16.897157 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:16.897141 2562 scope.go:117] "RemoveContainer" containerID="ff706fc31fdfde0dfcee8d074d465bc3342f07a514500afb46ca6ef174200f8f" Jun 07 09:11:17.898368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:17.898332 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5684bc65d4-vt78j" event={"ID":"45b898ad-9536-44fd-bbc1-ccd153fb9b09","Type":"ContainerStarted","Data":"a33e1a145406959287b135e90d62552bd157a92ce50fbcc7c5f16a0f8bf14d1a"} Jun 07 09:11:20.404817 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:20.404762 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:11:27.141146 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.141111 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-66bbd65f87-j4kdq"] Jun 07 09:11:27.142990 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.142975 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.145673 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.145649 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 07 09:11:27.145840 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.145752 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.146158 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.146136 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 07 09:11:27.146235 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.146163 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 07 09:11:27.146235 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.146142 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 07 09:11:27.146335 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.146142 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.147332 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.147314 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4q5p9\"" Jun 07 09:11:27.155832 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.155795 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66bbd65f87-j4kdq"] Jun 07 09:11:27.160243 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.160225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.160327 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.160248 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-stats-auth\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.160327 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.160274 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.160327 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.160296 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjtrt\" (UniqueName: \"kubernetes.io/projected/e897afd2-c08c-45a4-9466-f649cd07ef2f-kube-api-access-xjtrt\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.160422 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.160346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-default-certificate\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.228509 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.228490 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt"] Jun 07 09:11:27.230719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.230688 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.232078 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.232054 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr"] Jun 07 09:11:27.233560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.233539 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 07 09:11:27.233633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.233602 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 07 09:11:27.233697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.233630 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-npnmn\"" Jun 07 09:11:27.233898 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.233879 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hfnrb"] Jun 07 09:11:27.234007 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.233993 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.235732 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.235716 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:11:27.235846 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.235831 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.237466 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.237447 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jun 07 09:11:27.237569 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.237538 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.237673 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.237658 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.238872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.238551 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7rqff\"" Jun 07 09:11:27.238872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.238599 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.240380 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.240357 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jun 07 09:11:27.240477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.240400 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:11:27.240477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.240400 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jun 07 09:11:27.241141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241122 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt"] Jun 07 09:11:27.241338 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241296 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jun 07 09:11:27.241458 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241438 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-56nr8\"" Jun 07 09:11:27.241757 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241741 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 07 09:11:27.241861 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241827 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 07 09:11:27.242005 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.241986 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 07 09:11:27.242154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.242062 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sqrsh\"" Jun 07 09:11:27.250853 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.250833 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jun 07 09:11:27.250975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.250947 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 07 09:11:27.251547 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.251526 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr"] Jun 07 09:11:27.252633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.252611 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hfnrb"] Jun 07 09:11:27.260676 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260654 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.260754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260688 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.260754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260709 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z6s6f\" (UniqueName: \"kubernetes.io/projected/51eeb105-3b90-4033-b1fb-5c69ef46d98d-kube-api-access-z6s6f\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.260754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260727 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjtrt\" (UniqueName: \"kubernetes.io/projected/e897afd2-c08c-45a4-9466-f649cd07ef2f-kube-api-access-xjtrt\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.260754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260750 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-default-certificate\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.260780 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260838 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.260852 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.760834668 +0000 UTC m=+179.761839786 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : secret "router-metrics-certs-default" not found Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.260931 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.760920336 +0000 UTC m=+179.761925458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260933 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2p7g2\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.260985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.260981 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-trusted-ca\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261011 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261056 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261092 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-stats-auth\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261118 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8afb1a67-475c-4eb6-95b9-0022fad8cc23-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261145 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9781214e-f19d-4d43-a164-156e453a26a8-serving-cert\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261225 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.261279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261256 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.261667 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgv6k\" (UniqueName: \"kubernetes.io/projected/9781214e-f19d-4d43-a164-156e453a26a8-kube-api-access-kgv6k\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.261667 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261316 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-config\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.261667 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.261347 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.263421 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.263401 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-default-certificate\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.263491 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.263450 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-stats-auth\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.265335 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.265315 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:11:27.274524 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.274504 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjtrt\" (UniqueName: \"kubernetes.io/projected/e897afd2-c08c-45a4-9466-f649cd07ef2f-kube-api-access-xjtrt\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.362610 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362574 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.362610 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z6s6f\" (UniqueName: \"kubernetes.io/projected/51eeb105-3b90-4033-b1fb-5c69ef46d98d-kube-api-access-z6s6f\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.362707 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362762 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.362772 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.862758207 +0000 UTC m=+179.863763312 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362792 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2p7g2\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362831 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-trusted-ca\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.362894 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.362858 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.362922 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.362963 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls podName:51eeb105-3b90-4033-b1fb-5c69ef46d98d nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.86295137 +0000 UTC m=+179.863956487 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gclqr" (UID: "51eeb105-3b90-4033-b1fb-5c69ef46d98d") : secret "samples-operator-tls" not found Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363043 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363081 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8afb1a67-475c-4eb6-95b9-0022fad8cc23-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363107 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363137 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9781214e-f19d-4d43-a164-156e453a26a8-serving-cert\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363187 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363365 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363415 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363439 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgv6k\" (UniqueName: \"kubernetes.io/projected/9781214e-f19d-4d43-a164-156e453a26a8-kube-api-access-kgv6k\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-config\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363506 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.363612 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:11:27.363622 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.363626 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d7f597694-pdzf5: secret "image-registry-tls" not found Jun 07 09:11:27.363985 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.363667 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls podName:879e0229-0fbc-4f39-a124-28186bc15960 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:27.863653243 +0000 UTC m=+179.864658345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls") pod "image-registry-6d7f597694-pdzf5" (UID: "879e0229-0fbc-4f39-a124-28186bc15960") : secret "image-registry-tls" not found Jun 07 09:11:27.363985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.363780 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-trusted-ca\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.364090 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.364059 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.364279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.364257 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/9781214e-f19d-4d43-a164-156e453a26a8-config\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.364356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.364286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/8afb1a67-475c-4eb6-95b9-0022fad8cc23-nginx-conf\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.365547 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.365519 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.365650 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.365632 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.365957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.365940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/9781214e-f19d-4d43-a164-156e453a26a8-serving-cert\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.373933 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.373901 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z6s6f\" (UniqueName: \"kubernetes.io/projected/51eeb105-3b90-4033-b1fb-5c69ef46d98d-kube-api-access-z6s6f\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.373933 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.373920 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.374074 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.373979 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2p7g2\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.374340 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.374322 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgv6k\" (UniqueName: \"kubernetes.io/projected/9781214e-f19d-4d43-a164-156e453a26a8-kube-api-access-kgv6k\") pod \"console-operator-668fdbf85c-hfnrb\" (UID: \"9781214e-f19d-4d43-a164-156e453a26a8\") " pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.434256 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.434233 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:27.482006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.481982 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:27.556610 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.556584 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:27.675063 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.675033 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-668fdbf85c-hfnrb"] Jun 07 09:11:27.677856 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:27.677828 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9781214e_f19d_4d43_a164_156e453a26a8.slice/crio-df0d60bedacc8d69d0fd8696ff3f6ede4cd0964d0bd49a0357f50841a39edf8f WatchSource:0}: Error finding container df0d60bedacc8d69d0fd8696ff3f6ede4cd0964d0bd49a0357f50841a39edf8f: Status 404 returned error can't find the container with id df0d60bedacc8d69d0fd8696ff3f6ede4cd0964d0bd49a0357f50841a39edf8f Jun 07 09:11:27.767431 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.767373 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.767529 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.767449 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:27.767578 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.767527 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.767511263 +0000 UTC m=+180.768516364 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:27.767578 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.767557 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 07 09:11:27.767667 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.767612 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.767595021 +0000 UTC m=+180.768600125 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : secret "router-metrics-certs-default" not found Jun 07 09:11:27.868125 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.868095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:27.868288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.868139 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:27.868288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.868165 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:27.868288 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868249 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:27.868288 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868256 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:11:27.868288 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868280 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d7f597694-pdzf5: secret "image-registry-tls" not found Jun 07 09:11:27.868480 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868295 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.868281218 +0000 UTC m=+180.869286318 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:27.868480 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868321 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls podName:879e0229-0fbc-4f39-a124-28186bc15960 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.868309088 +0000 UTC m=+180.869314211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls") pod "image-registry-6d7f597694-pdzf5" (UID: "879e0229-0fbc-4f39-a124-28186bc15960") : secret "image-registry-tls" not found Jun 07 09:11:27.868480 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868254 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 07 09:11:27.868480 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:27.868353 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls podName:51eeb105-3b90-4033-b1fb-5c69ef46d98d nodeName:}" failed. No retries permitted until 2026-06-07 09:11:28.868346341 +0000 UTC m=+180.869351489 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gclqr" (UID: "51eeb105-3b90-4033-b1fb-5c69ef46d98d") : secret "samples-operator-tls" not found Jun 07 09:11:27.929000 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:27.928969 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" event={"ID":"9781214e-f19d-4d43-a164-156e453a26a8","Type":"ContainerStarted","Data":"df0d60bedacc8d69d0fd8696ff3f6ede4cd0964d0bd49a0357f50841a39edf8f"} Jun 07 09:11:28.775523 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:28.775477 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:28.776015 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:28.775584 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:28.776015 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.775629 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.77561031 +0000 UTC m=+182.776615411 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:28.776015 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.775689 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 07 09:11:28.776015 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.775748 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.775729978 +0000 UTC m=+182.776735096 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : secret "router-metrics-certs-default" not found Jun 07 09:11:28.876502 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:28.876464 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:28.876657 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:28.876523 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:28.876657 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:28.876572 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:28.876657 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876619 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:11:28.876657 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876641 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d7f597694-pdzf5: secret "image-registry-tls" not found Jun 07 09:11:28.876657 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876621 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 07 09:11:28.876829 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876658 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:28.876829 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876690 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls podName:879e0229-0fbc-4f39-a124-28186bc15960 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.876675457 +0000 UTC m=+182.877680558 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls") pod "image-registry-6d7f597694-pdzf5" (UID: "879e0229-0fbc-4f39-a124-28186bc15960") : secret "image-registry-tls" not found Jun 07 09:11:28.876829 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876702 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.876696624 +0000 UTC m=+182.877701724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:28.876829 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:28.876713 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls podName:51eeb105-3b90-4033-b1fb-5c69ef46d98d nodeName:}" failed. No retries permitted until 2026-06-07 09:11:30.876707357 +0000 UTC m=+182.877712458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gclqr" (UID: "51eeb105-3b90-4033-b1fb-5c69ef46d98d") : secret "samples-operator-tls" not found Jun 07 09:11:29.936187 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:29.936165 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/0.log" Jun 07 09:11:29.936483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:29.936200 2562 generic.go:356] "Generic (PLEG): container finished" podID="9781214e-f19d-4d43-a164-156e453a26a8" containerID="9a2324e772a449ea788e783e4c8e7a653e2a9ef3602f767a2c03d5eec511c429" exitCode=255 Jun 07 09:11:29.936483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:29.936235 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" event={"ID":"9781214e-f19d-4d43-a164-156e453a26a8","Type":"ContainerDied","Data":"9a2324e772a449ea788e783e4c8e7a653e2a9ef3602f767a2c03d5eec511c429"} Jun 07 09:11:29.936483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:29.936453 2562 scope.go:117] "RemoveContainer" containerID="9a2324e772a449ea788e783e4c8e7a653e2a9ef3602f767a2c03d5eec511c429" Jun 07 09:11:29.972440 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:29.972413 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:30.792591 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.792550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:30.792766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.792640 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:30.792766 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.792704 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 07 09:11:30.792880 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.792769 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.792756592 +0000 UTC m=+186.793761692 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:30.792880 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.792786 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.792780012 +0000 UTC m=+186.793785113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : secret "router-metrics-certs-default" not found Jun 07 09:11:30.893175 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.893134 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:30.893346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.893192 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:30.893346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.893218 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:30.893346 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893278 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 07 09:11:30.893346 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893327 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:30.893346 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893348 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls podName:51eeb105-3b90-4033-b1fb-5c69ef46d98d nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.893330248 +0000 UTC m=+186.894335366 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gclqr" (UID: "51eeb105-3b90-4033-b1fb-5c69ef46d98d") : secret "samples-operator-tls" not found Jun 07 09:11:30.893536 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893369 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.893357603 +0000 UTC m=+186.894362704 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:30.893536 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893328 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:11:30.893536 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893384 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d7f597694-pdzf5: secret "image-registry-tls" not found Jun 07 09:11:30.893536 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.893410 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls podName:879e0229-0fbc-4f39-a124-28186bc15960 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:34.893403869 +0000 UTC m=+186.894408970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls") pod "image-registry-6d7f597694-pdzf5" (UID: "879e0229-0fbc-4f39-a124-28186bc15960") : secret "image-registry-tls" not found Jun 07 09:11:30.940673 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.940644 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:11:30.941064 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.941005 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/0.log" Jun 07 09:11:30.941064 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.941037 2562 generic.go:356] "Generic (PLEG): container finished" podID="9781214e-f19d-4d43-a164-156e453a26a8" containerID="fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97" exitCode=255 Jun 07 09:11:30.941140 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.941117 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" event={"ID":"9781214e-f19d-4d43-a164-156e453a26a8","Type":"ContainerDied","Data":"fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97"} Jun 07 09:11:30.941193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.941181 2562 scope.go:117] "RemoveContainer" containerID="9a2324e772a449ea788e783e4c8e7a653e2a9ef3602f767a2c03d5eec511c429" Jun 07 09:11:30.941423 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:30.941408 2562 scope.go:117] "RemoveContainer" containerID="fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97" Jun 07 09:11:30.941600 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:30.941582 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hfnrb_openshift-console-operator(9781214e-f19d-4d43-a164-156e453a26a8)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" podUID="9781214e-f19d-4d43-a164-156e453a26a8" Jun 07 09:11:31.924906 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:31.924875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:31.944628 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:31.944606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:11:31.944965 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:31.944913 2562 scope.go:117] "RemoveContainer" containerID="fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97" Jun 07 09:11:31.945063 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:31.945046 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hfnrb_openshift-console-operator(9781214e-f19d-4d43-a164-156e453a26a8)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" podUID="9781214e-f19d-4d43-a164-156e453a26a8" Jun 07 09:11:32.373490 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:32.373414 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:32.467799 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:32.467765 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:32.568388 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:32.568362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:33.205287 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:33.205254 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:33.646335 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:33.646256 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:33.706756 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:33.706730 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:33.919131 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:33.919101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:34.019604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.019579 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:34.024575 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.024560 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vrbp2_91483ade-61fa-4480-8667-b5f93faa14c4/dns-node-resolver/0.log" Jun 07 09:11:34.624536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.624511 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j8gj8_875807c0-7e1f-4b5d-b2e9-7a5665aa0e07/node-ca/0.log" Jun 07 09:11:34.822348 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.822302 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:34.822529 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.822409 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:34.822529 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.822457 2562 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jun 07 09:11:34.822529 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.822521 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.822507022 +0000 UTC m=+194.823512126 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : secret "router-metrics-certs-default" not found Jun 07 09:11:34.822662 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.822560 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.822544868 +0000 UTC m=+194.823549968 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:34.923100 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.923052 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:34.923236 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.923171 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:34.923236 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923203 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:34.923236 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.923213 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:34.923350 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923270 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.923249427 +0000 UTC m=+194.924254531 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:34.923350 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923312 2562 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jun 07 09:11:34.923418 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923363 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls podName:51eeb105-3b90-4033-b1fb-5c69ef46d98d nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.923351427 +0000 UTC m=+194.924356532 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls") pod "cluster-samples-operator-854f8b96b6-gclqr" (UID: "51eeb105-3b90-4033-b1fb-5c69ef46d98d") : secret "samples-operator-tls" not found Jun 07 09:11:34.923418 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923314 2562 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 07 09:11:34.923418 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923381 2562 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6d7f597694-pdzf5: secret "image-registry-tls" not found Jun 07 09:11:34.923418 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:34.923409 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls podName:879e0229-0fbc-4f39-a124-28186bc15960 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:42.923397765 +0000 UTC m=+194.924402866 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls") pod "image-registry-6d7f597694-pdzf5" (UID: "879e0229-0fbc-4f39-a124-28186bc15960") : secret "image-registry-tls" not found Jun 07 09:11:34.995374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:34.995347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:35.047988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:35.047959 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:35.424817 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:35.424791 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4z8nf_7b820800-1a37-4938-9cd6-0cb826052e09/serve-healthcheck-canary/0.log" Jun 07 09:11:35.547111 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:35.547062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:36.454451 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:36.454416 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.173346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.173301 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.470345 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.470267 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.520561 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.520531 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.557543 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.557515 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:37.557648 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.557584 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:37.557939 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.557925 2562 scope.go:117] "RemoveContainer" containerID="fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97" Jun 07 09:11:37.558095 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:37.558076 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-668fdbf85c-hfnrb_openshift-console-operator(9781214e-f19d-4d43-a164-156e453a26a8)\"" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" podUID="9781214e-f19d-4d43-a164-156e453a26a8" Jun 07 09:11:37.580544 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.580510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.713473 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.713427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.766556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.766493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:37.861693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:37.861666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:38.866394 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:38.866357 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:39.069574 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:39.069540 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:39.467140 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:39.467101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:39.528606 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:39.528578 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:39.671704 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:39.671659 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:40.075509 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:40.075475 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:40.125893 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:40.125863 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:40.382213 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:40.382135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:40.865669 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:40.865638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.053596 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.053570 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.102115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.102085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.267394 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.267366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.468266 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.468228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.667269 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.667237 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.720537 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.720510 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:41.869351 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:41.869316 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:42.811379 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.811347 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:42.887444 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.887410 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:42.887611 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.887510 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:42.887611 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:42.887572 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle podName:e897afd2-c08c-45a4-9466-f649cd07ef2f nodeName:}" failed. No retries permitted until 2026-06-07 09:11:58.887555712 +0000 UTC m=+210.888560813 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle") pod "router-default-66bbd65f87-j4kdq" (UID: "e897afd2-c08c-45a4-9466-f649cd07ef2f") : configmap references non-existent config key: service-ca.crt Jun 07 09:11:42.889800 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.889782 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e897afd2-c08c-45a4-9466-f649cd07ef2f-metrics-certs\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:42.988314 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.988292 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:42.988419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.988330 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:42.988419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.988351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:42.988509 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:42.988424 2562 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 07 09:11:42.988552 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:11:42.988525 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert podName:8afb1a67-475c-4eb6-95b9-0022fad8cc23 nodeName:}" failed. No retries permitted until 2026-06-07 09:11:58.988508273 +0000 UTC m=+210.989513374 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert") pod "networking-console-plugin-d5d5bb78b-zmsvt" (UID: "8afb1a67-475c-4eb6-95b9-0022fad8cc23") : secret "networking-console-plugin-cert" not found Jun 07 09:11:42.990611 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.990582 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/51eeb105-3b90-4033-b1fb-5c69ef46d98d-samples-operator-tls\") pod \"cluster-samples-operator-854f8b96b6-gclqr\" (UID: \"51eeb105-3b90-4033-b1fb-5c69ef46d98d\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:42.990708 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:42.990615 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"image-registry-6d7f597694-pdzf5\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:43.071333 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.071235 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.151136 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.151114 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-7rqff\"" Jun 07 09:11:43.158503 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.158480 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" Jun 07 09:11:43.173239 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.173220 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-sqrsh\"" Jun 07 09:11:43.181671 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.181650 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:43.266720 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.266689 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.288593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.288572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr"] Jun 07 09:11:43.310076 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.310053 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:11:43.312817 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:43.312780 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod879e0229_0fbc_4f39_a124_28186bc15960.slice/crio-ec82989f35d62ff7122292351fb3c5f04aeceb112e75719425eff808d297fc63 WatchSource:0}: Error finding container ec82989f35d62ff7122292351fb3c5f04aeceb112e75719425eff808d297fc63: Status 404 returned error can't find the container with id ec82989f35d62ff7122292351fb3c5f04aeceb112e75719425eff808d297fc63 Jun 07 09:11:43.470397 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.470366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.653575 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.653497 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.712862 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.712830 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.864407 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.864366 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:43.979669 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.979632 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" event={"ID":"51eeb105-3b90-4033-b1fb-5c69ef46d98d","Type":"ContainerStarted","Data":"6a1ee77954f4c51f68f6a57b0847f41f39613785a9775c0a087d8583a4211b0d"} Jun 07 09:11:43.981507 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.981472 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" event={"ID":"879e0229-0fbc-4f39-a124-28186bc15960","Type":"ContainerStarted","Data":"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f"} Jun 07 09:11:43.981636 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.981518 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" event={"ID":"879e0229-0fbc-4f39-a124-28186bc15960","Type":"ContainerStarted","Data":"ec82989f35d62ff7122292351fb3c5f04aeceb112e75719425eff808d297fc63"} Jun 07 09:11:43.981636 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:43.981616 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:11:44.000535 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:44.000489 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" podStartSLOduration=17.000471688 podStartE2EDuration="17.000471688s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:43.999295795 +0000 UTC m=+196.000300920" watchObservedRunningTime="2026-06-07 09:11:44.000471688 +0000 UTC m=+196.001476815" Jun 07 09:11:44.867023 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:44.866985 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:45.335960 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.335916 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:45.467420 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.467394 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:45.667478 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.667441 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:45.864749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.864672 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:45.989599 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.989569 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" event={"ID":"51eeb105-3b90-4033-b1fb-5c69ef46d98d","Type":"ContainerStarted","Data":"050d58907a68b37c740ac99d6e20a4ba0d80e40f31e0eb8551aa4470e0e9ea84"} Jun 07 09:11:45.989971 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:45.989623 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" event={"ID":"51eeb105-3b90-4033-b1fb-5c69ef46d98d","Type":"ContainerStarted","Data":"33a82949d4e19bcd2c0bf635abaf4a8d40d070e4a18ef49402ed75deef0d1165"} Jun 07 09:11:46.006725 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:46.006678 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-854f8b96b6-gclqr" podStartSLOduration=17.255600821 podStartE2EDuration="19.006662967s" totalImagesPullingTime="1.751062146s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:46.004638829 +0000 UTC m=+198.005643952" watchObservedRunningTime="2026-06-07 09:11:46.006662967 +0000 UTC m=+198.007668090" Jun 07 09:11:46.866551 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:46.866513 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:47.062426 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:47.062390 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:47.466670 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:47.466637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:47.665125 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:47.665095 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:47.864863 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:47.864772 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:51.405532 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:51.405500 2562 scope.go:117] "RemoveContainer" containerID="fb51dd68cbc1efd9f1f05db75b4c5dc3421a9656082c399ec1c8192ea4c33a97" Jun 07 09:11:52.009209 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.009182 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:11:52.009373 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.009261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" event={"ID":"9781214e-f19d-4d43-a164-156e453a26a8","Type":"ContainerStarted","Data":"d19ec5b58037510474cb4fa5dc11ff76bdc487094a894d1f4e3f355d4c44acdf"} Jun 07 09:11:52.009516 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.009496 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:52.025629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.025577 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" podStartSLOduration=22.998185175 podStartE2EDuration="25.025561934s" totalImagesPullingTime="2.027376759s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:52.02537026 +0000 UTC m=+204.026375382" watchObservedRunningTime="2026-06-07 09:11:52.025561934 +0000 UTC m=+204.026567058" Jun 07 09:11:52.642352 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.642324 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-668fdbf85c-hfnrb" Jun 07 09:11:52.821991 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.821957 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:52.922109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.922078 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:52.972671 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:52.972645 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.112730 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.112701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.298465 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.298392 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.492440 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.492407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.690902 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.690873 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.743651 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.743624 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:53.892062 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:53.892029 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:54.201451 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.201423 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dj54p"] Jun 07 09:11:54.204548 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.204531 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.207453 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.207433 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 07 09:11:54.208936 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.208918 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 07 09:11:54.208936 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.208933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 07 09:11:54.209089 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.208963 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 07 09:11:54.209089 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.208933 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-lk99w\"" Jun 07 09:11:54.212707 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.212685 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dj54p"] Jun 07 09:11:54.254922 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.254899 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:11:54.269587 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.269561 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-crio-socket\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.269731 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.269703 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-data-volume\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.269886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.269768 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.269886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.269829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7tn6z\" (UniqueName: \"kubernetes.io/projected/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-api-access-7tn6z\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.269886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.269859 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.282853 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.282832 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56d8457687-n48gs"] Jun 07 09:11:54.285719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.285701 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.296879 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.296858 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d8457687-n48gs"] Jun 07 09:11:54.371066 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371038 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzfnl\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-kube-api-access-vzfnl\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-data-volume\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371122 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371157 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7tn6z\" (UniqueName: \"kubernetes.io/projected/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-api-access-7tn6z\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371185 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371216 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1727d6ab-acde-42bc-b86f-0e98d5949119-ca-trust-extracted\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-image-registry-private-configuration\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371296 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-crio-socket\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371322 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-bound-sa-token\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371361 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-data-volume\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371383 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-certificates\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371381 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-crio-socket\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.371698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371411 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-installation-pull-secrets\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371438 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-trusted-ca\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371473 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-tls\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.371893 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.371701 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.373377 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.373360 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.379900 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.379880 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7tn6z\" (UniqueName: \"kubernetes.io/projected/3e48adb9-4c2c-4f3c-ad6d-20029700f62a-kube-api-access-7tn6z\") pod \"insights-runtime-extractor-dj54p\" (UID: \"3e48adb9-4c2c-4f3c-ad6d-20029700f62a\") " pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.394501 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.394474 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:54.472483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472417 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1727d6ab-acde-42bc-b86f-0e98d5949119-ca-trust-extracted\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472462 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-image-registry-private-configuration\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472611 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472485 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-bound-sa-token\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472676 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472651 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-certificates\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472724 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472698 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-installation-pull-secrets\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472740 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-trusted-ca\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472822 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472783 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-tls\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472878 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472847 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzfnl\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-kube-api-access-vzfnl\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.472878 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.472849 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/1727d6ab-acde-42bc-b86f-0e98d5949119-ca-trust-extracted\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.473652 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.473593 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-certificates\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.473963 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.473940 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1727d6ab-acde-42bc-b86f-0e98d5949119-trusted-ca\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.475115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.475089 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-image-registry-private-configuration\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.475753 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.475735 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-registry-tls\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.475948 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.475929 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/1727d6ab-acde-42bc-b86f-0e98d5949119-installation-pull-secrets\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.481628 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.481604 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-bound-sa-token\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.484384 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.484362 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzfnl\" (UniqueName: \"kubernetes.io/projected/1727d6ab-acde-42bc-b86f-0e98d5949119-kube-api-access-vzfnl\") pod \"image-registry-56d8457687-n48gs\" (UID: \"1727d6ab-acde-42bc-b86f-0e98d5949119\") " pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.494675 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.494654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:54.513913 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.513896 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dj54p" Jun 07 09:11:54.594141 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.594113 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:54.599667 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.599642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:54.637214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.637186 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dj54p"] Jun 07 09:11:54.640313 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:54.640279 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e48adb9_4c2c_4f3c_ad6d_20029700f62a.slice/crio-3e323f7ef56593aa559676cc7dd6dfdc92169d572bdb9700100b5fdd375be28a WatchSource:0}: Error finding container 3e323f7ef56593aa559676cc7dd6dfdc92169d572bdb9700100b5fdd375be28a: Status 404 returned error can't find the container with id 3e323f7ef56593aa559676cc7dd6dfdc92169d572bdb9700100b5fdd375be28a Jun 07 09:11:54.704074 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.704045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:54.728451 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.728428 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56d8457687-n48gs"] Jun 07 09:11:54.730476 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:54.730442 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1727d6ab_acde_42bc_b86f_0e98d5949119.slice/crio-7c9375fb473d7f24b885fc86b146841d8377aa5a40a9c594368a4b22180f2209 WatchSource:0}: Error finding container 7c9375fb473d7f24b885fc86b146841d8377aa5a40a9c594368a4b22180f2209: Status 404 returned error can't find the container with id 7c9375fb473d7f24b885fc86b146841d8377aa5a40a9c594368a4b22180f2209 Jun 07 09:11:54.836767 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:54.836725 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:55.019303 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.019223 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj54p" event={"ID":"3e48adb9-4c2c-4f3c-ad6d-20029700f62a","Type":"ContainerStarted","Data":"a1dc1c45445cc1a89efa40ca3d942eb76f7d8908a5df46232c1d62925856e923"} Jun 07 09:11:55.019303 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.019288 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj54p" event={"ID":"3e48adb9-4c2c-4f3c-ad6d-20029700f62a","Type":"ContainerStarted","Data":"3e323f7ef56593aa559676cc7dd6dfdc92169d572bdb9700100b5fdd375be28a"} Jun 07 09:11:55.020597 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.020565 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d8457687-n48gs" event={"ID":"1727d6ab-acde-42bc-b86f-0e98d5949119","Type":"ContainerStarted","Data":"47a3bc366b3729751041ca1de470469edc477d71df9621ae492c498bbc94973c"} Jun 07 09:11:55.020729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.020607 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d8457687-n48gs" event={"ID":"1727d6ab-acde-42bc-b86f-0e98d5949119","Type":"ContainerStarted","Data":"7c9375fb473d7f24b885fc86b146841d8377aa5a40a9c594368a4b22180f2209"} Jun 07 09:11:55.020729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.020707 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:11:55.038971 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.038926 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podStartSLOduration=1.038912896 podStartE2EDuration="1.038912896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:55.037915304 +0000 UTC m=+207.038920427" watchObservedRunningTime="2026-06-07 09:11:55.038912896 +0000 UTC m=+207.039918010" Jun 07 09:11:55.317439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.317349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:55.558455 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.558427 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:55.606130 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:55.606074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:56.026709 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:56.026674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj54p" event={"ID":"3e48adb9-4c2c-4f3c-ad6d-20029700f62a","Type":"ContainerStarted","Data":"7f84ba7c676da8d3852e9d911c59e9e59624aeb942a2abbb00cb9589b94a9a3a"} Jun 07 09:11:56.211460 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:56.211423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:56.264536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:56.264496 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:56.339131 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:56.339063 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:11:57.032569 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:57.032531 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dj54p" event={"ID":"3e48adb9-4c2c-4f3c-ad6d-20029700f62a","Type":"ContainerStarted","Data":"32afc5a6bde8fb797183b2bf2e2cae157a6390eebb396f19840db8439512b3f7"} Jun 07 09:11:57.052273 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:57.052227 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dj54p" podStartSLOduration=0.997588437 podStartE2EDuration="3.052213291s" totalImagesPullingTime="2.054624854s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:11:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:11:57.050278187 +0000 UTC m=+209.051283312" watchObservedRunningTime="2026-06-07 09:11:57.052213291 +0000 UTC m=+209.053218414" Jun 07 09:11:58.909975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:58.909935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:58.910645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:58.910620 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e897afd2-c08c-45a4-9466-f649cd07ef2f-service-ca-bundle\") pod \"router-default-66bbd65f87-j4kdq\" (UID: \"e897afd2-c08c-45a4-9466-f649cd07ef2f\") " pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:58.955140 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:58.955116 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-4q5p9\"" Jun 07 09:11:58.962130 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:58.962112 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:11:59.010656 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.010627 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:59.013710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.013679 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/8afb1a67-475c-4eb6-95b9-0022fad8cc23-networking-console-plugin-cert\") pod \"networking-console-plugin-d5d5bb78b-zmsvt\" (UID: \"8afb1a67-475c-4eb6-95b9-0022fad8cc23\") " pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:59.045373 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.045354 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-npnmn\"" Jun 07 09:11:59.053331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.053303 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" Jun 07 09:11:59.086927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.086902 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-66bbd65f87-j4kdq"] Jun 07 09:11:59.089668 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:59.089642 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode897afd2_c08c_45a4_9466_f649cd07ef2f.slice/crio-1d1bb7d8ffa47340265687b8a3f9eb2a70d02b8212e5f32426cd611d0cf3979a WatchSource:0}: Error finding container 1d1bb7d8ffa47340265687b8a3f9eb2a70d02b8212e5f32426cd611d0cf3979a: Status 404 returned error can't find the container with id 1d1bb7d8ffa47340265687b8a3f9eb2a70d02b8212e5f32426cd611d0cf3979a Jun 07 09:11:59.178861 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:11:59.178836 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt"] Jun 07 09:11:59.181479 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:11:59.181446 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8afb1a67_475c_4eb6_95b9_0022fad8cc23.slice/crio-0c4152cfa220f2b1181061e6398f5b538ca0165ab683bc7ad57ce37039eb401a WatchSource:0}: Error finding container 0c4152cfa220f2b1181061e6398f5b538ca0165ab683bc7ad57ce37039eb401a: Status 404 returned error can't find the container with id 0c4152cfa220f2b1181061e6398f5b538ca0165ab683bc7ad57ce37039eb401a Jun 07 09:12:00.045078 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.045034 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" event={"ID":"8afb1a67-475c-4eb6-95b9-0022fad8cc23","Type":"ContainerStarted","Data":"0c4152cfa220f2b1181061e6398f5b538ca0165ab683bc7ad57ce37039eb401a"} Jun 07 09:12:00.046637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.046611 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" event={"ID":"e897afd2-c08c-45a4-9466-f649cd07ef2f","Type":"ContainerStarted","Data":"8ca672670fe7642f9367f08cc9b4f9cce625fef9bfb675d072860fa32e4fcd3d"} Jun 07 09:12:00.046761 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.046656 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" event={"ID":"e897afd2-c08c-45a4-9466-f649cd07ef2f","Type":"ContainerStarted","Data":"1d1bb7d8ffa47340265687b8a3f9eb2a70d02b8212e5f32426cd611d0cf3979a"} Jun 07 09:12:00.065307 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.065263 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" podStartSLOduration=33.065246633 podStartE2EDuration="33.065246633s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:00.065215158 +0000 UTC m=+212.066220282" watchObservedRunningTime="2026-06-07 09:12:00.065246633 +0000 UTC m=+212.066251746" Jun 07 09:12:00.962819 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.962763 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:12:00.965303 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:00.965280 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:12:01.051183 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.051155 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" event={"ID":"8afb1a67-475c-4eb6-95b9-0022fad8cc23","Type":"ContainerStarted","Data":"d841c5cf5bd97993f3e7f53a37180d8a3ebe8a02bdfed02bc87b7a5dbf0e25f3"} Jun 07 09:12:01.051537 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.051404 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:12:01.052440 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.052420 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-66bbd65f87-j4kdq" Jun 07 09:12:01.066645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.066602 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d5d5bb78b-zmsvt" podStartSLOduration=32.85202351 podStartE2EDuration="34.066586517s" totalImagesPullingTime="1.214563007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:11:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:01.066229915 +0000 UTC m=+213.067235037" watchObservedRunningTime="2026-06-07 09:12:01.066586517 +0000 UTC m=+213.067591641" Jun 07 09:12:01.155982 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.155947 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:01.212777 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.212744 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:01.963589 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:01.963557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:02.036874 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:02.036843 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:02.156413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:02.156378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:02.298134 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:02.298049 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:04.189272 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:04.189240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:04.248427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:04.248403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:04.260054 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:04.260028 2562 patch_prober.go:28] interesting pod/image-registry-6d7f597694-pdzf5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:04.260154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:04.260069 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:04.822566 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:04.822530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:05.762912 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:05.762881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:06.964973 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:06.964939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:08.330518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.330493 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:08.543284 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.543247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:08.596115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.596050 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-tz66r"] Jun 07 09:12:08.599341 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.599322 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.602268 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.602243 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 07 09:12:08.602594 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.602574 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 07 09:12:08.602648 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.602636 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 07 09:12:08.603911 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.603890 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 07 09:12:08.603911 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.603903 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 07 09:12:08.604096 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.603905 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 07 09:12:08.604096 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.603963 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pc6hm\"" Jun 07 09:12:08.629322 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.629294 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:08.687543 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687518 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687551 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687575 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-textfile\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687717 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687669 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-metrics-client-ca\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687717 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687708 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-root\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687823 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687823 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687770 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-wtmp\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687925 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687831 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbnrk\" (UniqueName: \"kubernetes.io/projected/7854433d-36c6-40a9-8c5f-26bfed42b8d2-kube-api-access-bbnrk\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.687925 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.687881 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-sys\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.711290 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.711265 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:08.788284 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788262 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-sys\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788357 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788309 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788357 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788340 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788445 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788374 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-textfile\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788445 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788377 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-sys\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788445 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788426 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-metrics-client-ca\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788544 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-root\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788606 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788581 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788649 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788611 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-wtmp\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788649 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788641 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbnrk\" (UniqueName: \"kubernetes.io/projected/7854433d-36c6-40a9-8c5f-26bfed42b8d2-kube-api-access-bbnrk\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788738 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788655 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-root\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788738 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:12:08.788715 2562 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 07 09:12:08.788851 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788771 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-wtmp\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.788851 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:12:08.788793 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls podName:7854433d-36c6-40a9-8c5f-26bfed42b8d2 nodeName:}" failed. No retries permitted until 2026-06-07 09:12:09.28877389 +0000 UTC m=+221.289779010 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls") pod "node-exporter-tz66r" (UID: "7854433d-36c6-40a9-8c5f-26bfed42b8d2") : secret "node-exporter-tls" not found Jun 07 09:12:08.788986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788963 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-accelerators-collector-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.789034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.788976 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7854433d-36c6-40a9-8c5f-26bfed42b8d2-metrics-client-ca\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.789034 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.789027 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-textfile\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.790766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.790747 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.797287 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.797261 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbnrk\" (UniqueName: \"kubernetes.io/projected/7854433d-36c6-40a9-8c5f-26bfed42b8d2-kube-api-access-bbnrk\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:08.943436 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:08.943407 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.045338 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.045312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.143258 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.143230 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.277414 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.277345 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.294598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.294577 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:09.296846 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.296825 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7854433d-36c6-40a9-8c5f-26bfed42b8d2-node-exporter-tls\") pod \"node-exporter-tz66r\" (UID: \"7854433d-36c6-40a9-8c5f-26bfed42b8d2\") " pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:09.399694 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.399673 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.509226 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.509201 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-tz66r" Jun 07 09:12:09.520222 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:12:09.520191 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7854433d_36c6_40a9_8c5f_26bfed42b8d2.slice/crio-7695763a22d7da4fd54cbd6648abfcd3289c0f7817411e98b554d37918bf7afe WatchSource:0}: Error finding container 7695763a22d7da4fd54cbd6648abfcd3289c0f7817411e98b554d37918bf7afe: Status 404 returned error can't find the container with id 7695763a22d7da4fd54cbd6648abfcd3289c0f7817411e98b554d37918bf7afe Jun 07 09:12:09.560021 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.559964 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.642995 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.642971 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.760817 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.760774 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:09.961742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:09.961707 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:10.081941 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:10.081909 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz66r" event={"ID":"7854433d-36c6-40a9-8c5f-26bfed42b8d2","Type":"ContainerStarted","Data":"7695763a22d7da4fd54cbd6648abfcd3289c0f7817411e98b554d37918bf7afe"} Jun 07 09:12:10.566183 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:10.566149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:10.627646 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:10.627574 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:11.087007 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.086971 2562 generic.go:356] "Generic (PLEG): container finished" podID="7854433d-36c6-40a9-8c5f-26bfed42b8d2" containerID="ac33abb29c7715b9a90892050b8eedcfcd4bb69e4e0a444eed4fd2d5e17fc978" exitCode=0 Jun 07 09:12:11.087168 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.087042 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz66r" event={"ID":"7854433d-36c6-40a9-8c5f-26bfed42b8d2","Type":"ContainerDied","Data":"ac33abb29c7715b9a90892050b8eedcfcd4bb69e4e0a444eed4fd2d5e17fc978"} Jun 07 09:12:11.498609 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.498577 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:11.545885 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.545857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:11.784342 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.784268 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:11.873772 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.873739 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:11.970916 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:11.970887 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:12.092886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:12.092790 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz66r" event={"ID":"7854433d-36c6-40a9-8c5f-26bfed42b8d2","Type":"ContainerStarted","Data":"a605825685e071ccfc15189313d96943da6878ca185cb0c195cac8ffe534f81b"} Jun 07 09:12:12.092997 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:12.092896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-tz66r" event={"ID":"7854433d-36c6-40a9-8c5f-26bfed42b8d2","Type":"ContainerStarted","Data":"6ad3822c41e2324252cd40be109442c6a2a8e8a4469501bfbc5675ca1561b75a"} Jun 07 09:12:12.115633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:12.115588 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-tz66r" podStartSLOduration=3.223398054 podStartE2EDuration="4.115575298s" totalImagesPullingTime="829.125389ms" totalInitContainerRuntime="63.051855ms" isStatefulPod=true podCreationTimestamp="2026-06-07 09:12:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:12.113275744 +0000 UTC m=+224.114280894" watchObservedRunningTime="2026-06-07 09:12:12.115575298 +0000 UTC m=+224.116580421" Jun 07 09:12:13.003771 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.003736 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-85c9fd84d4-lltd4"] Jun 07 09:12:13.006510 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.006493 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.009865 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.009842 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 07 09:12:13.009983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.009876 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 07 09:12:13.009983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.009841 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-k9xtd\"" Jun 07 09:12:13.009983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.009848 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 07 09:12:13.010779 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.010752 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 07 09:12:13.010916 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.010882 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-c6sn63nm7hhkp\"" Jun 07 09:12:13.018852 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.018833 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85c9fd84d4-lltd4"] Jun 07 09:12:13.125937 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.125908 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-audit-log\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.125937 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.125940 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-metrics-server-audit-profiles\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.126154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.125962 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-tls\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.126154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.125992 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9r2l\" (UniqueName: \"kubernetes.io/projected/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-kube-api-access-z9r2l\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.126154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.126043 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-client-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.126154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.126078 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-client-certs\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.126154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.126099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227402 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227363 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-tls\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227567 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227415 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9r2l\" (UniqueName: \"kubernetes.io/projected/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-kube-api-access-z9r2l\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227567 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-client-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227567 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227470 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-client-certs\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227567 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227493 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.227963 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227915 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-audit-log\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.228131 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.227982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-metrics-server-audit-profiles\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.228380 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.228352 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-audit-log\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.228477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.228364 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.229209 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.229183 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-metrics-server-audit-profiles\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.230041 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.230022 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-tls\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.230137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.230043 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-client-ca-bundle\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.230263 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.230243 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-secret-metrics-server-client-certs\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.236125 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.236101 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9r2l\" (UniqueName: \"kubernetes.io/projected/2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd-kube-api-access-z9r2l\") pod \"metrics-server-85c9fd84d4-lltd4\" (UID: \"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd\") " pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.315950 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.315875 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:13.344009 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.343977 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:13.370232 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.370203 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6"] Jun 07 09:12:13.374709 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.374675 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.378841 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.377456 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 07 09:12:13.378841 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.377797 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-vnrhw\"" Jun 07 09:12:13.380040 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.380015 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6"] Jun 07 09:12:13.418432 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.418406 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:13.446457 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.446438 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-85c9fd84d4-lltd4"] Jun 07 09:12:13.448712 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:12:13.448687 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c6a610e_a35b_4c71_8d9c_80ba2f38a9cd.slice/crio-1bbf4d8f0513f27347ff8f547f2776fefa911d9e748e717844e02b6a55a8730d WatchSource:0}: Error finding container 1bbf4d8f0513f27347ff8f547f2776fefa911d9e748e717844e02b6a55a8730d: Status 404 returned error can't find the container with id 1bbf4d8f0513f27347ff8f547f2776fefa911d9e748e717844e02b6a55a8730d Jun 07 09:12:13.531380 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.531346 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-kube-api-access-w7tph\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.531488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.531435 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.632605 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.632550 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-kube-api-access-w7tph\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.632698 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.632610 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.632698 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:12:13.632695 2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 07 09:12:13.632765 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:12:13.632740 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert podName:d1611ab7-07a0-4f5b-a326-ff62f2ed2caf nodeName:}" failed. No retries permitted until 2026-06-07 09:12:14.132727254 +0000 UTC m=+226.133732356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert") pod "monitoring-plugin-76b68cc874-d8tg6" (UID: "d1611ab7-07a0-4f5b-a326-ff62f2ed2caf") : secret "monitoring-plugin-cert" not found Jun 07 09:12:13.641170 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.641147 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7tph\" (UniqueName: \"kubernetes.io/projected/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-kube-api-access-w7tph\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:13.916186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:13.916159 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:14.100906 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.100868 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" event={"ID":"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd","Type":"ContainerStarted","Data":"1bbf4d8f0513f27347ff8f547f2776fefa911d9e748e717844e02b6a55a8730d"} Jun 07 09:12:14.137012 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.136982 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:14.139644 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.139619 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/d1611ab7-07a0-4f5b-a326-ff62f2ed2caf-monitoring-plugin-cert\") pod \"monitoring-plugin-76b68cc874-d8tg6\" (UID: \"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf\") " pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:14.260135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.260065 2562 patch_prober.go:28] interesting pod/image-registry-6d7f597694-pdzf5 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:14.260135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.260119 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:14.288469 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.288443 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:14.423838 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.423719 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6"] Jun 07 09:12:14.598957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.598870 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:14.598957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.598928 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:14.760509 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:12:14.760478 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1611ab7_07a0_4f5b_a326_ff62f2ed2caf.slice/crio-71f228ada5367927d6da7af277ab42ec194c47cb690e38272a10fca97c17d03a WatchSource:0}: Error finding container 71f228ada5367927d6da7af277ab42ec194c47cb690e38272a10fca97c17d03a: Status 404 returned error can't find the container with id 71f228ada5367927d6da7af277ab42ec194c47cb690e38272a10fca97c17d03a Jun 07 09:12:14.788099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:14.788070 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:15.105983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:15.105887 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" event={"ID":"2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd","Type":"ContainerStarted","Data":"67454ea26bd9587cff37d9fde355b77a2a67498c58ff6b189248ef311ef1ac69"} Jun 07 09:12:15.107268 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:15.107233 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" event={"ID":"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf","Type":"ContainerStarted","Data":"71f228ada5367927d6da7af277ab42ec194c47cb690e38272a10fca97c17d03a"} Jun 07 09:12:15.123880 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:15.123838 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" podStartSLOduration=1.766757865 podStartE2EDuration="3.12382625s" totalImagesPullingTime="1.357068385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:15.122257543 +0000 UTC m=+227.123262665" watchObservedRunningTime="2026-06-07 09:12:15.12382625 +0000 UTC m=+227.124831370" Jun 07 09:12:16.031731 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:16.031689 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:16.031912 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:16.031739 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:17.115069 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.115035 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" event={"ID":"d1611ab7-07a0-4f5b-a326-ff62f2ed2caf","Type":"ContainerStarted","Data":"84a58f2e57f9335b1363c72328b405a44142a9287219c829b1848428a344a5a9"} Jun 07 09:12:17.115465 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.115277 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:17.119816 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.119779 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" Jun 07 09:12:17.131856 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.131795 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-76b68cc874-d8tg6" podStartSLOduration=2.294826729 podStartE2EDuration="4.131778019s" totalImagesPullingTime="1.83695129s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:17.130889108 +0000 UTC m=+229.131894230" watchObservedRunningTime="2026-06-07 09:12:17.131778019 +0000 UTC m=+229.132783142" Jun 07 09:12:17.223450 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.223420 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:12:17.226487 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.226469 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.229504 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229456 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 07 09:12:17.229629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229514 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-k2swh\"" Jun 07 09:12:17.229629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229516 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 07 09:12:17.229629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229519 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 07 09:12:17.229995 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229632 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 07 09:12:17.229995 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229468 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 07 09:12:17.229995 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.229468 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 07 09:12:17.230158 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.230129 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 07 09:12:17.235222 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.235163 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 07 09:12:17.236689 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.236671 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:12:17.371372 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371294 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371372 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371337 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371378 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371427 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5pbl\" (UniqueName: \"kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371523 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371670 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371564 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.371670 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.371595 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472198 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472168 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472327 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472206 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472327 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472242 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472327 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472360 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472400 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.472581 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.472448 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5pbl\" (UniqueName: \"kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.473168 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.473002 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.473168 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.473085 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.473336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.473173 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.473336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.473204 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.475043 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.475014 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.475147 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.475077 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.480529 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.480507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5pbl\" (UniqueName: \"kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl\") pod \"console-76777f67b6-hzf8t\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.536349 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.536314 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:17.658928 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:17.658855 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:12:17.661599 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:12:17.661567 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dcb3600_6940_48b4_af8c_755a27e8723f.slice/crio-3629005b521d9f26710f597fc12378a9ce22fee3d0324e3269a51377e854712b WatchSource:0}: Error finding container 3629005b521d9f26710f597fc12378a9ce22fee3d0324e3269a51377e854712b: Status 404 returned error can't find the container with id 3629005b521d9f26710f597fc12378a9ce22fee3d0324e3269a51377e854712b Jun 07 09:12:18.125588 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:18.125550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76777f67b6-hzf8t" event={"ID":"7dcb3600-6940-48b4-af8c-755a27e8723f","Type":"ContainerStarted","Data":"3629005b521d9f26710f597fc12378a9ce22fee3d0324e3269a51377e854712b"} Jun 07 09:12:19.274580 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:19.274524 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" containerID="cri-o://3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f" gracePeriod=30 Jun 07 09:12:20.127667 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.127636 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:20.180321 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.180291 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:20.334037 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.334011 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:12:20.398343 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398257 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398343 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398305 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2p7g2\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398343 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398321 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398343 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398346 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398659 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398372 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398659 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398393 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398659 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398415 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.398659 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398436 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted\") pod \"879e0229-0fbc-4f39-a124-28186bc15960\" (UID: \"879e0229-0fbc-4f39-a124-28186bc15960\") " Jun 07 09:12:20.399071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.398946 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:20.399151 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.399108 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:12:20.400632 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.400591 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2" (OuterVolumeSpecName: "kube-api-access-2p7g2") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "kube-api-access-2p7g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:12:20.400800 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.400777 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:12:20.401081 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.401040 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:12:20.401162 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.401124 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:12:20.401277 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.401258 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:12:20.407573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.407550 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "879e0229-0fbc-4f39-a124-28186bc15960" (UID: "879e0229-0fbc-4f39-a124-28186bc15960"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:12:20.499758 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499735 2562 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-installation-pull-secrets\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499758 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499758 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2p7g2\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-kube-api-access-2p7g2\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499767 2562 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-registry-tls\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499776 2562 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/879e0229-0fbc-4f39-a124-28186bc15960-bound-sa-token\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499784 2562 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/879e0229-0fbc-4f39-a124-28186bc15960-image-registry-private-configuration\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499793 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-trusted-ca\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499801 2562 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/879e0229-0fbc-4f39-a124-28186bc15960-registry-certificates\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:20.499909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:20.499826 2562 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/879e0229-0fbc-4f39-a124-28186bc15960-ca-trust-extracted\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:12:21.138193 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.138152 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76777f67b6-hzf8t" event={"ID":"7dcb3600-6940-48b4-af8c-755a27e8723f","Type":"ContainerStarted","Data":"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba"} Jun 07 09:12:21.139314 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.139283 2562 generic.go:356] "Generic (PLEG): container finished" podID="879e0229-0fbc-4f39-a124-28186bc15960" containerID="3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f" exitCode=0 Jun 07 09:12:21.139427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.139314 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" Jun 07 09:12:21.139427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.139366 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" event={"ID":"879e0229-0fbc-4f39-a124-28186bc15960","Type":"ContainerDied","Data":"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f"} Jun 07 09:12:21.139427 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.139407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6d7f597694-pdzf5" event={"ID":"879e0229-0fbc-4f39-a124-28186bc15960","Type":"ContainerDied","Data":"ec82989f35d62ff7122292351fb3c5f04aeceb112e75719425eff808d297fc63"} Jun 07 09:12:21.139525 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.139427 2562 scope.go:117] "RemoveContainer" containerID="3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f" Jun 07 09:12:21.147984 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.147966 2562 scope.go:117] "RemoveContainer" containerID="3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f" Jun 07 09:12:21.148252 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:12:21.148233 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f\": container with ID starting with 3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f not found: ID does not exist" containerID="3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f" Jun 07 09:12:21.148296 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.148262 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f"} err="failed to get container status \"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f\": rpc error: code = NotFound desc = could not find container \"3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f\": container with ID starting with 3c1330ab49854d17f63dcfe2f152fe34b5617ce0d51c119451ff34900003d39f not found: ID does not exist" Jun 07 09:12:21.157060 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.157020 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76777f67b6-hzf8t" podStartSLOduration=1.5788924469999999 podStartE2EDuration="4.157007748s" totalImagesPullingTime="2.578115301s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:12:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:21.155871979 +0000 UTC m=+233.156877102" watchObservedRunningTime="2026-06-07 09:12:21.157007748 +0000 UTC m=+233.158012942" Jun 07 09:12:21.170665 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.170641 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:12:21.173875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:21.173853 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6d7f597694-pdzf5"] Jun 07 09:12:22.412829 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:22.412784 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="879e0229-0fbc-4f39-a124-28186bc15960" path="/var/lib/kubelet/pods/879e0229-0fbc-4f39-a124-28186bc15960/volumes" Jun 07 09:12:24.316550 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:24.316520 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:24.372593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:24.372565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:24.599025 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:24.598947 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:24.599025 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:24.598994 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:26.030862 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:26.030831 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:26.031239 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:26.030876 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:27.536752 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:27.536717 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:27.537151 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:27.536774 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:12:27.538087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:27.538063 2562 patch_prober.go:28] interesting pod/console-76777f67b6-hzf8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 07 09:12:27.538203 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:27.538102 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 07 09:12:29.650482 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:29.650447 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:29.695096 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:29.695045 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:31.119053 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:31.119020 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:31.169349 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:31.169323 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:32.238336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:32.238305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:32.288037 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:32.288005 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:33.316650 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:33.316607 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:33.317029 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:33.316671 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:34.598178 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.598142 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:34.598559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.598204 2562 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:34.598559 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.598241 2562 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:12:34.598707 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.598683 2562 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"47a3bc366b3729751041ca1de470469edc477d71df9621ae492c498bbc94973c"} pod="openshift-image-registry/image-registry-56d8457687-n48gs" containerMessage="Container registry failed liveness probe, will be restarted" Jun 07 09:12:34.605940 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.605899 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:34.606144 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:34.606120 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:37.537262 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:37.537222 2562 patch_prober.go:28] interesting pod/console-76777f67b6-hzf8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 07 09:12:37.537633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:37.537268 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 07 09:12:39.085799 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:39.085754 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:39.134967 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:39.134938 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:40.248525 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:40.248496 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:12:40.250682 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:40.250665 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d72371b8-2fcd-44d3-8280-1b3af902f221-metrics-certs\") pod \"network-metrics-daemon-rkp7x\" (UID: \"d72371b8-2fcd-44d3-8280-1b3af902f221\") " pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:12:40.508890 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:40.508794 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jh2fl\"" Jun 07 09:12:40.516314 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:40.516298 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rkp7x" Jun 07 09:12:40.635492 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:40.635462 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rkp7x"] Jun 07 09:12:40.637433 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:12:40.637406 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd72371b8_2fcd_44d3_8280_1b3af902f221.slice/crio-32c0126d700153439f787ed9f06fdce16f653da8d792c903123ab5581e9ba345 WatchSource:0}: Error finding container 32c0126d700153439f787ed9f06fdce16f653da8d792c903123ab5581e9ba345: Status 404 returned error can't find the container with id 32c0126d700153439f787ed9f06fdce16f653da8d792c903123ab5581e9ba345 Jun 07 09:12:41.209095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:41.209058 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkp7x" event={"ID":"d72371b8-2fcd-44d3-8280-1b3af902f221","Type":"ContainerStarted","Data":"32c0126d700153439f787ed9f06fdce16f653da8d792c903123ab5581e9ba345"} Jun 07 09:12:42.214169 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:42.214126 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkp7x" event={"ID":"d72371b8-2fcd-44d3-8280-1b3af902f221","Type":"ContainerStarted","Data":"ca8558730ffc1715d1588969e28f2e0bdee6eeb8176868300d004d9a33e82a0d"} Jun 07 09:12:42.214519 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:42.214227 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rkp7x" event={"ID":"d72371b8-2fcd-44d3-8280-1b3af902f221","Type":"ContainerStarted","Data":"71763378e01dc5c82547a30a08a1dd9989fb1047bdf004b8516f6b144ab89c9f"} Jun 07 09:12:42.229643 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:42.229588 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rkp7x" podStartSLOduration=253.330630524 podStartE2EDuration="4m14.229570046s" totalImagesPullingTime="898.939522ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:08:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:12:42.229375973 +0000 UTC m=+254.230381098" watchObservedRunningTime="2026-06-07 09:12:42.229570046 +0000 UTC m=+254.230575169" Jun 07 09:12:42.319533 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:42.319506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:42.419921 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:42.419890 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:44.602701 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:44.602672 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:44.603147 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:44.602720 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:47.536998 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:47.536964 2562 patch_prober.go:28] interesting pod/console-76777f67b6-hzf8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 07 09:12:47.537374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:47.537036 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 07 09:12:48.775824 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:48.775777 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:48.822500 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:48.822469 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:53.321206 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:53.321179 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:53.324990 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:53.324966 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-85c9fd84d4-lltd4" Jun 07 09:12:54.603376 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:54.603347 2562 patch_prober.go:28] interesting pod/image-registry-56d8457687-n48gs container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jun 07 09:12:54.603746 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:54.603403 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:12:55.560277 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:55.560243 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:55.610331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:55.610306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:57.536708 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:57.536673 2562 patch_prober.go:28] interesting pod/console-76777f67b6-hzf8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 07 09:12:57.537105 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:57.536721 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 07 09:12:58.295452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:58.295420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:58.352306 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:58.352275 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:12:59.618590 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:12:59.618554 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56d8457687-n48gs" podUID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerName="registry" containerID="cri-o://47a3bc366b3729751041ca1de470469edc477d71df9621ae492c498bbc94973c" gracePeriod=30 Jun 07 09:13:01.280381 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:01.280350 2562 generic.go:356] "Generic (PLEG): container finished" podID="1727d6ab-acde-42bc-b86f-0e98d5949119" containerID="47a3bc366b3729751041ca1de470469edc477d71df9621ae492c498bbc94973c" exitCode=0 Jun 07 09:13:01.280733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:01.280423 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d8457687-n48gs" event={"ID":"1727d6ab-acde-42bc-b86f-0e98d5949119","Type":"ContainerDied","Data":"47a3bc366b3729751041ca1de470469edc477d71df9621ae492c498bbc94973c"} Jun 07 09:13:01.280733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:01.280495 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56d8457687-n48gs" event={"ID":"1727d6ab-acde-42bc-b86f-0e98d5949119","Type":"ContainerStarted","Data":"961167a4cf68da4268ae3a0ada3872f5d7074ce9238a123505db895508286fed"} Jun 07 09:13:01.280733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:01.280663 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:13:07.537174 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:07.537135 2562 patch_prober.go:28] interesting pod/console-76777f67b6-hzf8t container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" start-of-body= Jun 07 09:13:07.537645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:07.537200 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" probeResult="failure" output="Get \"https://10.134.0.18:8443/health\": dial tcp 10.134.0.18:8443: connect: connection refused" Jun 07 09:13:07.837886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:07.837764 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:07.883548 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:07.883504 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:09.869537 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:13:09.869480 2562 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-9pv2c" podUID="50a6ee97-e3e3-489c-b2f8-0f20de1fdd57" Jun 07 09:13:10.311126 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:10.311093 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:12.781404 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:12.781368 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:12.783636 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:12.783616 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/50a6ee97-e3e3-489c-b2f8-0f20de1fdd57-metrics-tls\") pod \"dns-default-9pv2c\" (UID: \"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57\") " pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:13.015334 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.015303 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xmvt5\"" Jun 07 09:13:13.022868 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.022850 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:13.144641 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.144615 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-9pv2c"] Jun 07 09:13:13.147337 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:13:13.147311 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a6ee97_e3e3_489c_b2f8_0f20de1fdd57.slice/crio-3b1e8ffc685faab3533682cd0d1a28ab91ee24bc4a016338c3c1e740f0b565ad WatchSource:0}: Error finding container 3b1e8ffc685faab3533682cd0d1a28ab91ee24bc4a016338c3c1e740f0b565ad: Status 404 returned error can't find the container with id 3b1e8ffc685faab3533682cd0d1a28ab91ee24bc4a016338c3c1e740f0b565ad Jun 07 09:13:13.327593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.327498 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9pv2c" event={"ID":"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57","Type":"ContainerStarted","Data":"3b1e8ffc685faab3533682cd0d1a28ab91ee24bc4a016338c3c1e740f0b565ad"} Jun 07 09:13:13.900742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.900695 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:13.956405 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:13.956362 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:14.268602 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:14.268565 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:14.317010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:14.316973 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:14.637886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:14.637851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:14.738718 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:14.738654 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:15.337574 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.337543 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9pv2c" event={"ID":"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57","Type":"ContainerStarted","Data":"dfeba6311e98abc9b584b70b7b7d6981c15d0c200923d6b2f1b90e9002d535df"} Jun 07 09:13:15.337979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.337595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-9pv2c" event={"ID":"50a6ee97-e3e3-489c-b2f8-0f20de1fdd57","Type":"ContainerStarted","Data":"19e5d4faaea4a5d4d3fdf862d5a1727b1a1cc8e2d6ed2b46430e0f1942aa67af"} Jun 07 09:13:15.337979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.337637 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:15.355517 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.355477 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-9pv2c" podStartSLOduration=251.91026205 podStartE2EDuration="4m13.355465381s" totalImagesPullingTime="1.445203331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:09:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:13:15.353637884 +0000 UTC m=+287.354643008" watchObservedRunningTime="2026-06-07 09:13:15.355465381 +0000 UTC m=+287.356470504" Jun 07 09:13:15.438570 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.438543 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:15.540056 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:15.540031 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:17.540763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:17.540736 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:13:17.544467 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:17.544441 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:13:17.699126 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:17.699096 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:17.796789 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:17.796704 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:17.848697 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:17.848662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:22.289664 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:22.289630 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-56d8457687-n48gs" Jun 07 09:13:22.411155 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:22.411112 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:22.512261 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:22.512227 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:22.612335 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:22.612251 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:24.643413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:24.643383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:24.695128 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:24.695101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:25.347160 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:25.347123 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-9pv2c" Jun 07 09:13:25.398076 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:25.398041 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:25.452356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:25.452328 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:25.995694 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:25.995658 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:26.094544 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.094519 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:26.165654 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.165619 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:13:26.166016 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.165998 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" Jun 07 09:13:26.166016 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.166018 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" Jun 07 09:13:26.166191 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.166100 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="879e0229-0fbc-4f39-a124-28186bc15960" containerName="registry" Jun 07 09:13:26.172001 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.171980 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.181148 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.181127 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:13:26.212627 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.212596 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:26.263678 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.263621 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:26.284944 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.284923 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285003 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.284950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285003 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.284969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285083 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.285062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.285092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.285108 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jw679\" (UniqueName: \"kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.285183 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.285174 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386400 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386378 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386406 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386423 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jw679\" (UniqueName: \"kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386467 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386504 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386528 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.386800 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.386554 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.387229 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.387198 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.387347 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.387274 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.387347 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.387285 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.387536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.387514 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.388992 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.388973 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.389086 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.389025 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.394619 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.394595 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jw679\" (UniqueName: \"kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679\") pod \"console-768fd7998b-vlcnv\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.483581 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.483554 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:26.608080 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:26.608047 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:13:26.610963 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:13:26.610934 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb2bf123_bf3c_4b4c_bfbd_ad8a55d792f9.slice/crio-19ede2fad0624cb608d16e0d245f2c47da20adc99481a0c43b3cef6e9ecc8861 WatchSource:0}: Error finding container 19ede2fad0624cb608d16e0d245f2c47da20adc99481a0c43b3cef6e9ecc8861: Status 404 returned error can't find the container with id 19ede2fad0624cb608d16e0d245f2c47da20adc99481a0c43b3cef6e9ecc8861 Jun 07 09:13:27.387124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:27.387083 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fd7998b-vlcnv" event={"ID":"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9","Type":"ContainerStarted","Data":"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96"} Jun 07 09:13:27.387488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:27.387141 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fd7998b-vlcnv" event={"ID":"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9","Type":"ContainerStarted","Data":"19ede2fad0624cb608d16e0d245f2c47da20adc99481a0c43b3cef6e9ecc8861"} Jun 07 09:13:27.404353 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:27.404308 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-768fd7998b-vlcnv" podStartSLOduration=1.404294942 podStartE2EDuration="1.404294942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:13:27.403064899 +0000 UTC m=+299.404070047" watchObservedRunningTime="2026-06-07 09:13:27.404294942 +0000 UTC m=+299.405300102" Jun 07 09:13:28.346940 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.346909 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:13:28.349176 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.349147 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:13:28.352279 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.352244 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:13:28.354998 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.354977 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:13:28.358930 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.358909 2562 kubelet.go:1648] "Image garbage collection succeeded" Jun 07 09:13:28.462050 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.462023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:28.562496 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:28.562472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:34.464346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:34.464315 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:34.553563 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:34.553528 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:36.484338 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:36.484308 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:36.484713 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:36.484354 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:36.489077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:36.489055 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:37.434216 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:37.434182 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:13:37.480241 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:37.480206 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:13:37.541702 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:37.541666 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:37.641469 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:37.641437 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:37.689243 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:37.689174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:40.560350 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:40.560314 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:40.610200 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:40.610171 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:55.568386 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:55.568352 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:13:55.668264 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:13:55.668240 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:02.500871 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.500822 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-76777f67b6-hzf8t" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" containerID="cri-o://0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba" gracePeriod=15 Jun 07 09:14:02.736314 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.736285 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:14:02.750394 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750369 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750428 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750464 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750500 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750530 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750572 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750614 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5pbl\" (UniqueName: \"kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl\") pod \"7dcb3600-6940-48b4-af8c-755a27e8723f\" (UID: \"7dcb3600-6940-48b4-af8c-755a27e8723f\") " Jun 07 09:14:02.750912 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750845 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config" (OuterVolumeSpecName: "console-config") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:14:02.750912 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750898 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:14:02.751048 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.750964 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca" (OuterVolumeSpecName: "service-ca") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:14:02.751289 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.751264 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:14:02.752783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.752760 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:14:02.752783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.752778 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:14:02.752919 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.752793 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl" (OuterVolumeSpecName: "kube-api-access-r5pbl") pod "7dcb3600-6940-48b4-af8c-755a27e8723f" (UID: "7dcb3600-6940-48b4-af8c-755a27e8723f"). InnerVolumeSpecName "kube-api-access-r5pbl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:14:02.851356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851320 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-oauth-serving-cert\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851352 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-trusted-ca-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851362 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-serving-cert\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851371 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7dcb3600-6940-48b4-af8c-755a27e8723f-console-oauth-config\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851382 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-service-ca\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851390 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5pbl\" (UniqueName: \"kubernetes.io/projected/7dcb3600-6940-48b4-af8c-755a27e8723f-kube-api-access-r5pbl\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:02.851573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:02.851398 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7dcb3600-6940-48b4-af8c-755a27e8723f-console-config\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:03.525556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.525520 2562 generic.go:356] "Generic (PLEG): container finished" podID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerID="0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba" exitCode=0 Jun 07 09:14:03.526077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.525584 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76777f67b6-hzf8t" Jun 07 09:14:03.526077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.525601 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76777f67b6-hzf8t" event={"ID":"7dcb3600-6940-48b4-af8c-755a27e8723f","Type":"ContainerDied","Data":"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba"} Jun 07 09:14:03.526077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.525675 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76777f67b6-hzf8t" event={"ID":"7dcb3600-6940-48b4-af8c-755a27e8723f","Type":"ContainerDied","Data":"3629005b521d9f26710f597fc12378a9ce22fee3d0324e3269a51377e854712b"} Jun 07 09:14:03.526077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.525696 2562 scope.go:117] "RemoveContainer" containerID="0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba" Jun 07 09:14:03.533955 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.533940 2562 scope.go:117] "RemoveContainer" containerID="0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba" Jun 07 09:14:03.534200 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:14:03.534183 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba\": container with ID starting with 0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba not found: ID does not exist" containerID="0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba" Jun 07 09:14:03.534245 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.534207 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba"} err="failed to get container status \"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba\": rpc error: code = NotFound desc = could not find container \"0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba\": container with ID starting with 0b83d7f414e09c0b32a3711682c18729fd3267ea436697ba377b351eb1559aba not found: ID does not exist" Jun 07 09:14:03.546199 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.546180 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:14:03.549732 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:03.549714 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76777f67b6-hzf8t"] Jun 07 09:14:04.409047 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:04.409012 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" path="/var/lib/kubelet/pods/7dcb3600-6940-48b4-af8c-755a27e8723f/volumes" Jun 07 09:14:15.708271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:15.708241 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:15.794286 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:15.794258 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:16.263172 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.263140 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75"] Jun 07 09:14:16.263396 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.263382 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" Jun 07 09:14:16.263396 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.263395 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" Jun 07 09:14:16.263471 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.263447 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="7dcb3600-6940-48b4-af8c-755a27e8723f" containerName="console" Jun 07 09:14:16.266438 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.266422 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.269353 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.269331 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:14:16.269503 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.269338 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:14:16.269503 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.269340 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:14:16.274115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.274093 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75"] Jun 07 09:14:16.342637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.342611 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.342754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.342653 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srt9x\" (UniqueName: \"kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.342754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.342676 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.443416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.443388 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srt9x\" (UniqueName: \"kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.443546 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.443424 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.443611 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.443547 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.443766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.443749 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.443838 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.443792 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.451531 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.451513 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-srt9x\" (UniqueName: \"kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.575721 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.575651 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:16.698780 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.698755 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75"] Jun 07 09:14:16.701318 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:14:16.701291 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4cc5b54_52f9_430c_b24f_f09de26f2a61.slice/crio-68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72 WatchSource:0}: Error finding container 68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72: Status 404 returned error can't find the container with id 68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72 Jun 07 09:14:16.703325 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:16.703308 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:14:17.577720 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:17.577685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" event={"ID":"b4cc5b54-52f9-430c-b24f-f09de26f2a61","Type":"ContainerStarted","Data":"68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72"} Jun 07 09:14:22.599858 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:22.599821 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerID="921fbc59709dd976c81ad078aa0ff951963a418e20c90826ad6bd16708912097" exitCode=0 Jun 07 09:14:22.600273 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:22.599895 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" event={"ID":"b4cc5b54-52f9-430c-b24f-f09de26f2a61","Type":"ContainerDied","Data":"921fbc59709dd976c81ad078aa0ff951963a418e20c90826ad6bd16708912097"} Jun 07 09:14:23.605603 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:23.605567 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerID="364d52b5ba49af03c2bf5f61d23bfc6d9c69ad757c07701d5741b07350847adf" exitCode=0 Jun 07 09:14:23.606023 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:23.605617 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" event={"ID":"b4cc5b54-52f9-430c-b24f-f09de26f2a61","Type":"ContainerDied","Data":"364d52b5ba49af03c2bf5f61d23bfc6d9c69ad757c07701d5741b07350847adf"} Jun 07 09:14:30.634425 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:30.634345 2562 generic.go:356] "Generic (PLEG): container finished" podID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerID="007161a588fcc4fa0ab33bbd29b91beb3d3f9a47de0cbbc4720a78f1bc468ab0" exitCode=0 Jun 07 09:14:30.634888 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:30.634419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" event={"ID":"b4cc5b54-52f9-430c-b24f-f09de26f2a61","Type":"ContainerDied","Data":"007161a588fcc4fa0ab33bbd29b91beb3d3f9a47de0cbbc4720a78f1bc468ab0"} Jun 07 09:14:31.762971 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.762948 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:31.867092 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.867060 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srt9x\" (UniqueName: \"kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x\") pod \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " Jun 07 09:14:31.867214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.867109 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util\") pod \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " Jun 07 09:14:31.867214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.867150 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle\") pod \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\" (UID: \"b4cc5b54-52f9-430c-b24f-f09de26f2a61\") " Jun 07 09:14:31.867985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.867955 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle" (OuterVolumeSpecName: "bundle") pod "b4cc5b54-52f9-430c-b24f-f09de26f2a61" (UID: "b4cc5b54-52f9-430c-b24f-f09de26f2a61"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:14:31.869257 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.869232 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x" (OuterVolumeSpecName: "kube-api-access-srt9x") pod "b4cc5b54-52f9-430c-b24f-f09de26f2a61" (UID: "b4cc5b54-52f9-430c-b24f-f09de26f2a61"). InnerVolumeSpecName "kube-api-access-srt9x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:14:31.871566 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.871544 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util" (OuterVolumeSpecName: "util") pod "b4cc5b54-52f9-430c-b24f-f09de26f2a61" (UID: "b4cc5b54-52f9-430c-b24f-f09de26f2a61"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:14:31.967584 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.967552 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srt9x\" (UniqueName: \"kubernetes.io/projected/b4cc5b54-52f9-430c-b24f-f09de26f2a61-kube-api-access-srt9x\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:31.967584 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.967582 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:31.967584 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:31.967592 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b4cc5b54-52f9-430c-b24f-f09de26f2a61-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:14:32.643225 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:32.643192 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" event={"ID":"b4cc5b54-52f9-430c-b24f-f09de26f2a61","Type":"ContainerDied","Data":"68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72"} Jun 07 09:14:32.643225 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:32.643213 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d6fn75" Jun 07 09:14:32.643478 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:32.643237 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="68a6e9c1bb35110745bb228823e8bb9ff17c7d23ed177dbd1ef46b1c49c49f72" Jun 07 09:14:37.200250 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:37.200221 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:38.016312 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:38.016280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:39.173535 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173508 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x"] Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173762 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="pull" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173772 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="pull" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173792 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="util" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173798 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="util" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173824 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="extract" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173830 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="extract" Jun 07 09:14:39.173949 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.173878 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4cc5b54-52f9-430c-b24f-f09de26f2a61" containerName="extract" Jun 07 09:14:39.175540 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.175525 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.178330 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.178308 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jun 07 09:14:39.178477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.178411 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-t78sx\"" Jun 07 09:14:39.178477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.178412 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:14:39.185958 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.185939 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x"] Jun 07 09:14:39.321078 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.321049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgsn6\" (UniqueName: \"kubernetes.io/projected/c3ddd022-f418-4116-8d5f-0c0029732529-kube-api-access-tgsn6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.321209 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.321092 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3ddd022-f418-4116-8d5f-0c0029732529-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.422300 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.422276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgsn6\" (UniqueName: \"kubernetes.io/projected/c3ddd022-f418-4116-8d5f-0c0029732529-kube-api-access-tgsn6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.422398 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.422328 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3ddd022-f418-4116-8d5f-0c0029732529-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.422669 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.422654 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c3ddd022-f418-4116-8d5f-0c0029732529-tmp\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.430979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.430921 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgsn6\" (UniqueName: \"kubernetes.io/projected/c3ddd022-f418-4116-8d5f-0c0029732529-kube-api-access-tgsn6\") pod \"cert-manager-operator-controller-manager-57757f7cfc-fml4x\" (UID: \"c3ddd022-f418-4116-8d5f-0c0029732529\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.484289 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.484270 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" Jun 07 09:14:39.606705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.606678 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x"] Jun 07 09:14:39.610567 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:14:39.610544 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ddd022_f418_4116_8d5f_0c0029732529.slice/crio-873cb833bc464fd842d0c2207e729604432a73b7f11bed8cf72c103247eb0163 WatchSource:0}: Error finding container 873cb833bc464fd842d0c2207e729604432a73b7f11bed8cf72c103247eb0163: Status 404 returned error can't find the container with id 873cb833bc464fd842d0c2207e729604432a73b7f11bed8cf72c103247eb0163 Jun 07 09:14:39.669621 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:39.669595 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" event={"ID":"c3ddd022-f418-4116-8d5f-0c0029732529","Type":"ContainerStarted","Data":"873cb833bc464fd842d0c2207e729604432a73b7f11bed8cf72c103247eb0163"} Jun 07 09:14:41.005754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:41.005717 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:41.051837 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:41.051792 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:41.679813 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:41.679767 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" event={"ID":"c3ddd022-f418-4116-8d5f-0c0029732529","Type":"ContainerStarted","Data":"1b8a6fdb1520a90bcc637e79671b4ee20875f8a017185d8b018a1577300706a9"} Jun 07 09:14:41.704831 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:41.704772 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-57757f7cfc-fml4x" podStartSLOduration=1.197107396 podStartE2EDuration="2.704755751s" totalImagesPullingTime="1.507648355s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:14:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:14:41.702106527 +0000 UTC m=+373.703111652" watchObservedRunningTime="2026-06-07 09:14:41.704755751 +0000 UTC m=+373.705760874" Jun 07 09:14:42.097115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:42.097032 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:42.236214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:42.236172 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:42.340961 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:42.340931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:45.566380 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:45.566349 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:45.949364 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:45.949327 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:47.722744 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.722713 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-plrbf"] Jun 07 09:14:47.724789 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.724773 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.727755 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.727732 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jun 07 09:14:47.727867 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.727742 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jun 07 09:14:47.729248 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.729224 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z4b75\"" Jun 07 09:14:47.733636 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.733614 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-plrbf"] Jun 07 09:14:47.889002 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.888959 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fck2h\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-kube-api-access-fck2h\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.889132 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.889079 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.989928 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.989845 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.989928 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.989892 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fck2h\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-kube-api-access-fck2h\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.998452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.998424 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-bound-sa-token\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:47.998634 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:47.998614 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fck2h\" (UniqueName: \"kubernetes.io/projected/b39f3d62-f233-4b6d-b863-c26302befdf6-kube-api-access-fck2h\") pod \"cert-manager-cainjector-5d99449ffd-plrbf\" (UID: \"b39f3d62-f233-4b6d-b863-c26302befdf6\") " pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:48.035165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.035143 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" Jun 07 09:14:48.156852 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.156751 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5d99449ffd-plrbf"] Jun 07 09:14:48.159483 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:14:48.159451 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb39f3d62_f233_4b6d_b863_c26302befdf6.slice/crio-eab088be3223c69262e2ddd525c11b097cd12c4622ba80d3168aa72a6ac3f95b WatchSource:0}: Error finding container eab088be3223c69262e2ddd525c11b097cd12c4622ba80d3168aa72a6ac3f95b: Status 404 returned error can't find the container with id eab088be3223c69262e2ddd525c11b097cd12c4622ba80d3168aa72a6ac3f95b Jun 07 09:14:48.265074 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.265016 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:48.604135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.604057 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:48.668756 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.668729 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:48.708625 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:48.708591 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" event={"ID":"b39f3d62-f233-4b6d-b863-c26302befdf6","Type":"ContainerStarted","Data":"eab088be3223c69262e2ddd525c11b097cd12c4622ba80d3168aa72a6ac3f95b"} Jun 07 09:14:50.718861 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:50.718822 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" event={"ID":"b39f3d62-f233-4b6d-b863-c26302befdf6","Type":"ContainerStarted","Data":"b40a3f8b63b6d9488db271de32f3b990f48fc1c338096cf4fd49614ca81cb5c7"} Jun 07 09:14:50.736115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:50.736064 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5d99449ffd-plrbf" podStartSLOduration=1.3958841180000001 podStartE2EDuration="3.736045844s" totalImagesPullingTime="2.340161726s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:14:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:14:50.735134398 +0000 UTC m=+382.736139521" watchObservedRunningTime="2026-06-07 09:14:50.736045844 +0000 UTC m=+382.737050970" Jun 07 09:14:51.952954 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:51.952920 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:52.053682 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:52.053648 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:53.174911 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:53.174881 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:14:58.013768 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.013739 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29"] Jun 07 09:14:58.015934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.015918 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.018978 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.018956 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:14:58.019074 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.018986 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:14:58.020363 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.020348 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:14:58.024705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.024684 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29"] Jun 07 09:14:58.168527 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.168501 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fxqm\" (UniqueName: \"kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.168638 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.168535 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.168638 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.168568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.269007 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.268945 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.269007 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.268990 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.269134 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.269029 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fxqm\" (UniqueName: \"kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.269294 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.269276 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.269349 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.269331 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.277876 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.277850 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fxqm\" (UniqueName: \"kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.326224 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.326199 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:14:58.454663 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.454639 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29"] Jun 07 09:14:58.456959 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:14:58.456921 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af9d117_ece1_4eab_ba37_279c7bdc9d5d.slice/crio-6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241 WatchSource:0}: Error finding container 6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241: Status 404 returned error can't find the container with id 6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241 Jun 07 09:14:58.750777 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.750744 2562 generic.go:356] "Generic (PLEG): container finished" podID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerID="e2d335d00b900eb74f5b2cddc90e4444e124ce85a7602314ed2562a68f35a573" exitCode=0 Jun 07 09:14:58.750946 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.750789 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerDied","Data":"e2d335d00b900eb74f5b2cddc90e4444e124ce85a7602314ed2562a68f35a573"} Jun 07 09:14:58.750946 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:14:58.750851 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerStarted","Data":"6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241"} Jun 07 09:15:00.762704 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:00.762674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerStarted","Data":"06c8ccf8286bb2447efc4cccf16cf2f049ee532f855821a7835553ddb5bdeec2"} Jun 07 09:15:01.767781 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:01.767743 2562 generic.go:356] "Generic (PLEG): container finished" podID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerID="06c8ccf8286bb2447efc4cccf16cf2f049ee532f855821a7835553ddb5bdeec2" exitCode=0 Jun 07 09:15:01.768271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:01.767832 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerDied","Data":"06c8ccf8286bb2447efc4cccf16cf2f049ee532f855821a7835553ddb5bdeec2"} Jun 07 09:15:02.773880 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:02.773842 2562 generic.go:356] "Generic (PLEG): container finished" podID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerID="a88fd3962d98d668f3498d364db39cc3683af76cbb7d8c9d9866330a9e42a3c1" exitCode=0 Jun 07 09:15:02.774258 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:02.773896 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerDied","Data":"a88fd3962d98d668f3498d364db39cc3683af76cbb7d8c9d9866330a9e42a3c1"} Jun 07 09:15:03.900921 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:03.900898 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:15:04.010495 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.010466 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle\") pod \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " Jun 07 09:15:04.010634 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.010504 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fxqm\" (UniqueName: \"kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm\") pod \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " Jun 07 09:15:04.010634 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.010537 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util\") pod \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\" (UID: \"8af9d117-ece1-4eab-ba37-279c7bdc9d5d\") " Jun 07 09:15:04.010907 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.010884 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle" (OuterVolumeSpecName: "bundle") pod "8af9d117-ece1-4eab-ba37-279c7bdc9d5d" (UID: "8af9d117-ece1-4eab-ba37-279c7bdc9d5d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:15:04.012495 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.012473 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm" (OuterVolumeSpecName: "kube-api-access-5fxqm") pod "8af9d117-ece1-4eab-ba37-279c7bdc9d5d" (UID: "8af9d117-ece1-4eab-ba37-279c7bdc9d5d"). InnerVolumeSpecName "kube-api-access-5fxqm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:15:04.081178 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.081123 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util" (OuterVolumeSpecName: "util") pod "8af9d117-ece1-4eab-ba37-279c7bdc9d5d" (UID: "8af9d117-ece1-4eab-ba37-279c7bdc9d5d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:15:04.111536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.111508 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:04.111536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.111533 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:04.111710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.111547 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fxqm\" (UniqueName: \"kubernetes.io/projected/8af9d117-ece1-4eab-ba37-279c7bdc9d5d-kube-api-access-5fxqm\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:04.785336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.785258 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" Jun 07 09:15:04.785336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.785266 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ffrd29" event={"ID":"8af9d117-ece1-4eab-ba37-279c7bdc9d5d","Type":"ContainerDied","Data":"6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241"} Jun 07 09:15:04.785336 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:04.785309 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6ef00c5af0af6052ec66df9b02e98887af079f37eb710542c93b88178f8b1241" Jun 07 09:15:08.888899 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:08.888865 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:08.990935 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:08.990904 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:10.723311 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723274 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88"] Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723520 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="extract" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723532 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="extract" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723555 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="pull" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723561 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="pull" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723569 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="util" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723574 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="util" Jun 07 09:15:10.723693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.723618 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af9d117-ece1-4eab-ba37-279c7bdc9d5d" containerName="extract" Jun 07 09:15:10.725414 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.725398 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.728273 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.728252 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 07 09:15:10.729757 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.729738 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-pw7fd\"" Jun 07 09:15:10.729882 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.729740 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 07 09:15:10.733078 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.733059 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88"] Jun 07 09:15:10.862799 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.862760 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af803b05-4240-4341-8bb9-087c399cedb7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.862996 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.862846 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z85wq\" (UniqueName: \"kubernetes.io/projected/af803b05-4240-4341-8bb9-087c399cedb7-kube-api-access-z85wq\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.963906 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.963872 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z85wq\" (UniqueName: \"kubernetes.io/projected/af803b05-4240-4341-8bb9-087c399cedb7-kube-api-access-z85wq\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.964084 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.963966 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af803b05-4240-4341-8bb9-087c399cedb7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.964287 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.964268 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/af803b05-4240-4341-8bb9-087c399cedb7-tmp\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:10.973086 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:10.973066 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z85wq\" (UniqueName: \"kubernetes.io/projected/af803b05-4240-4341-8bb9-087c399cedb7-kube-api-access-z85wq\") pod \"openshift-lws-operator-fd8ccff4c-njt88\" (UID: \"af803b05-4240-4341-8bb9-087c399cedb7\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:11.035109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:11.035048 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" Jun 07 09:15:11.159600 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:11.159572 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88"] Jun 07 09:15:11.162966 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:11.162937 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaf803b05_4240_4341_8bb9_087c399cedb7.slice/crio-32911db3bb4abb47d650c1ea8205b63b7aea2be0c4a0dd96cd5001f39599aea4 WatchSource:0}: Error finding container 32911db3bb4abb47d650c1ea8205b63b7aea2be0c4a0dd96cd5001f39599aea4: Status 404 returned error can't find the container with id 32911db3bb4abb47d650c1ea8205b63b7aea2be0c4a0dd96cd5001f39599aea4 Jun 07 09:15:11.815457 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:11.815417 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" event={"ID":"af803b05-4240-4341-8bb9-087c399cedb7","Type":"ContainerStarted","Data":"32911db3bb4abb47d650c1ea8205b63b7aea2be0c4a0dd96cd5001f39599aea4"} Jun 07 09:15:13.825989 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:13.825953 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" event={"ID":"af803b05-4240-4341-8bb9-087c399cedb7","Type":"ContainerStarted","Data":"dddc8a2058520e128aa0da8ef18cb784727db625975e66420ae7c23c2887a4b5"} Jun 07 09:15:13.841588 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:13.841543 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-njt88" podStartSLOduration=2.18389965 podStartE2EDuration="3.841529326s" totalImagesPullingTime="1.657629676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:13.840728409 +0000 UTC m=+405.841733532" watchObservedRunningTime="2026-06-07 09:15:13.841529326 +0000 UTC m=+405.842534447" Jun 07 09:15:14.500649 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:14.500616 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:14.602154 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:14.602116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:16.497479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.497446 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj"] Jun 07 09:15:16.499719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.499698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.502710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.502684 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:15:16.502839 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.502757 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:15:16.504211 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.504197 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:15:16.509589 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.509567 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj"] Jun 07 09:15:16.603147 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.603109 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.603298 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.603150 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bz8rb\" (UniqueName: \"kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.603298 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.603258 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.608428 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.608403 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:16.703784 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.703758 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bz8rb\" (UniqueName: \"kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.703909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.703828 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.703909 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.703860 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.704153 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.704139 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.704207 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.704189 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.712059 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.712035 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bz8rb\" (UniqueName: \"kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb\") pod \"410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.809073 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.808976 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:16.928767 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:16.928609 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj"] Jun 07 09:15:16.931388 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:16.931356 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099adf43_6c62_4b7a_9785_eab4e81ed112.slice/crio-e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947 WatchSource:0}: Error finding container e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947: Status 404 returned error can't find the container with id e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947 Jun 07 09:15:17.844079 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:17.844044 2562 generic.go:356] "Generic (PLEG): container finished" podID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerID="c92942df05793a2cdd4f62c82ba8164294d3cb3d8982b11ee2dc3461e250f77e" exitCode=0 Jun 07 09:15:17.844452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:17.844132 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" event={"ID":"099adf43-6c62-4b7a-9785-eab4e81ed112","Type":"ContainerDied","Data":"c92942df05793a2cdd4f62c82ba8164294d3cb3d8982b11ee2dc3461e250f77e"} Jun 07 09:15:17.844452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:17.844179 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" event={"ID":"099adf43-6c62-4b7a-9785-eab4e81ed112","Type":"ContainerStarted","Data":"e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947"} Jun 07 09:15:18.850428 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:18.850356 2562 generic.go:356] "Generic (PLEG): container finished" podID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerID="6795ebc75132aca37e4826ed4e279b734fa0237b9d742daf2d8e09ab6b69dfba" exitCode=0 Jun 07 09:15:18.850990 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:18.850444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" event={"ID":"099adf43-6c62-4b7a-9785-eab4e81ed112","Type":"ContainerDied","Data":"6795ebc75132aca37e4826ed4e279b734fa0237b9d742daf2d8e09ab6b69dfba"} Jun 07 09:15:19.857162 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:19.857121 2562 generic.go:356] "Generic (PLEG): container finished" podID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerID="9d969f89d558cab575ec20378fd1f7141e79ce1763da5e0c4be448d22c9ff099" exitCode=0 Jun 07 09:15:19.857522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:19.857208 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" event={"ID":"099adf43-6c62-4b7a-9785-eab4e81ed112","Type":"ContainerDied","Data":"9d969f89d558cab575ec20378fd1f7141e79ce1763da5e0c4be448d22c9ff099"} Jun 07 09:15:20.986977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:20.986956 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:21.035653 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.035612 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bz8rb\" (UniqueName: \"kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb\") pod \"099adf43-6c62-4b7a-9785-eab4e81ed112\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " Jun 07 09:15:21.035785 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.035678 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util\") pod \"099adf43-6c62-4b7a-9785-eab4e81ed112\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " Jun 07 09:15:21.035785 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.035728 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle\") pod \"099adf43-6c62-4b7a-9785-eab4e81ed112\" (UID: \"099adf43-6c62-4b7a-9785-eab4e81ed112\") " Jun 07 09:15:21.036415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.036382 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle" (OuterVolumeSpecName: "bundle") pod "099adf43-6c62-4b7a-9785-eab4e81ed112" (UID: "099adf43-6c62-4b7a-9785-eab4e81ed112"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:15:21.037708 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.037673 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb" (OuterVolumeSpecName: "kube-api-access-bz8rb") pod "099adf43-6c62-4b7a-9785-eab4e81ed112" (UID: "099adf43-6c62-4b7a-9785-eab4e81ed112"). InnerVolumeSpecName "kube-api-access-bz8rb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:15:21.041277 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.041246 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util" (OuterVolumeSpecName: "util") pod "099adf43-6c62-4b7a-9785-eab4e81ed112" (UID: "099adf43-6c62-4b7a-9785-eab4e81ed112"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:15:21.136389 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.136328 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bz8rb\" (UniqueName: \"kubernetes.io/projected/099adf43-6c62-4b7a-9785-eab4e81ed112-kube-api-access-bz8rb\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:21.136389 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.136352 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:21.136389 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.136362 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/099adf43-6c62-4b7a-9785-eab4e81ed112-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:15:21.866735 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.866705 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" Jun 07 09:15:21.866735 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.866708 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/410e69f41462d9d02a48f76ad929b5f41ed146182df7ab38cf9ce4e1cdlm4gj" event={"ID":"099adf43-6c62-4b7a-9785-eab4e81ed112","Type":"ContainerDied","Data":"e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947"} Jun 07 09:15:21.866972 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:21.866755 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e19b1c69fcf5f9f9388a1b8c1ad3f14fd4a76a3ef1b075e472121424bc6de947" Jun 07 09:15:24.813307 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:24.813270 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:24.885529 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:24.885501 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:32.240422 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:32.240391 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:32.334670 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:32.334640 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:32.857047 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:32.857013 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.085577 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.085546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.184584 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.184551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.213505 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213477 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c"] Jun 07 09:15:33.213767 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213754 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="pull" Jun 07 09:15:33.213844 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213769 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="pull" Jun 07 09:15:33.213844 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213818 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="util" Jun 07 09:15:33.213844 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213827 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="util" Jun 07 09:15:33.213844 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213835 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="extract" Jun 07 09:15:33.213844 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213842 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="extract" Jun 07 09:15:33.213997 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.213894 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="099adf43-6c62-4b7a-9785-eab4e81ed112" containerName="extract" Jun 07 09:15:33.218861 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.218844 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.221892 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.221866 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 07 09:15:33.222055 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.222020 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 07 09:15:33.222227 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.222202 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 07 09:15:33.222334 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.222233 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gmnw6\"" Jun 07 09:15:33.222334 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.222278 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 07 09:15:33.231142 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.231122 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c"] Jun 07 09:15:33.333711 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.333685 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.334072 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.333725 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.334072 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.333751 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtgmh\" (UniqueName: \"kubernetes.io/projected/c2b95379-5adc-4ecf-a056-bc77dcb807d2-kube-api-access-qtgmh\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.334072 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.333776 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.334072 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.333874 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.335690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.335668 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.434975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.434913 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.434975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.434948 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.434975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.434968 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtgmh\" (UniqueName: \"kubernetes.io/projected/c2b95379-5adc-4ecf-a056-bc77dcb807d2-kube-api-access-qtgmh\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.435257 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.435093 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.435257 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.435181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.435488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.435469 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-manifests\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.437186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.437161 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c2b95379-5adc-4ecf-a056-bc77dcb807d2-tmp\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.437403 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.437386 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.437536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.437516 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c2b95379-5adc-4ecf-a056-bc77dcb807d2-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.441171 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.441149 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.453367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.453344 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtgmh\" (UniqueName: \"kubernetes.io/projected/c2b95379-5adc-4ecf-a056-bc77dcb807d2-kube-api-access-qtgmh\") pod \"opendatahub-operator-controller-manager-84c76964bf-7b88c\" (UID: \"c2b95379-5adc-4ecf-a056-bc77dcb807d2\") " pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.529570 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.529547 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:33.536633 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.536607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.641095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.639062 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.668467 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.668444 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c"] Jun 07 09:15:33.669595 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:33.669565 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2b95379_5adc_4ecf_a056_bc77dcb807d2.slice/crio-b82836163d3cf5e1f5edcecc3ddaf6dc6c1affff10bf4e66694b0e1affa53abd WatchSource:0}: Error finding container b82836163d3cf5e1f5edcecc3ddaf6dc6c1affff10bf4e66694b0e1affa53abd: Status 404 returned error can't find the container with id b82836163d3cf5e1f5edcecc3ddaf6dc6c1affff10bf4e66694b0e1affa53abd Jun 07 09:15:33.736626 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.736561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:33.918398 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:33.918363 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" event={"ID":"c2b95379-5adc-4ecf-a056-bc77dcb807d2","Type":"ContainerStarted","Data":"b82836163d3cf5e1f5edcecc3ddaf6dc6c1affff10bf4e66694b0e1affa53abd"} Jun 07 09:15:36.934381 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:36.934348 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" event={"ID":"c2b95379-5adc-4ecf-a056-bc77dcb807d2","Type":"ContainerStarted","Data":"a65584b8ed6902c165832ba8b29e92539564ca8e4ab1342d5f24bb6bd0175cef"} Jun 07 09:15:37.940373 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:37.940343 2562 generic.go:356] "Generic (PLEG): container finished" podID="c2b95379-5adc-4ecf-a056-bc77dcb807d2" containerID="a65584b8ed6902c165832ba8b29e92539564ca8e4ab1342d5f24bb6bd0175cef" exitCode=0 Jun 07 09:15:37.940766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:37.940429 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" event={"ID":"c2b95379-5adc-4ecf-a056-bc77dcb807d2","Type":"ContainerDied","Data":"a65584b8ed6902c165832ba8b29e92539564ca8e4ab1342d5f24bb6bd0175cef"} Jun 07 09:15:38.946791 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:38.946754 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" event={"ID":"c2b95379-5adc-4ecf-a056-bc77dcb807d2","Type":"ContainerStarted","Data":"3baac2d31b88a2277eb8ff60d24129daf43a8b722310801860ab8b87185d9a5c"} Jun 07 09:15:38.947267 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:38.946856 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:38.975553 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:38.975506 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" podStartSLOduration=1.7578681600000001 podStartE2EDuration="5.975492147s" totalImagesPullingTime="3.245632767s" totalInitContainerRuntime="971.99122ms" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:38.974167089 +0000 UTC m=+430.975172212" watchObservedRunningTime="2026-06-07 09:15:38.975492147 +0000 UTC m=+430.976497269" Jun 07 09:15:49.953858 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:49.953825 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c76964bf-7b88c" Jun 07 09:15:51.629165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:51.629122 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:51.752266 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:51.752232 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.051248 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.051216 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.147331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.147305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.306117 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.306042 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.348944 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.348908 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl"] Jun 07 09:15:52.354207 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.354183 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.357165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.357141 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 07 09:15:52.357165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.357156 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 07 09:15:52.363254 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.363224 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl"] Jun 07 09:15:52.484346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.484311 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tmp\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.484346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.484348 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4tq7\" (UniqueName: \"kubernetes.io/projected/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-kube-api-access-d4tq7\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.484572 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.484446 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.566119 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.566052 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hw6j8"] Jun 07 09:15:52.568242 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.568227 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.571211 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.571193 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 07 09:15:52.571337 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.571319 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-6snld\"" Jun 07 09:15:52.578763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.578743 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hw6j8"] Jun 07 09:15:52.584997 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.584975 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.585101 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.585027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tmp\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.585101 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.585056 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4tq7\" (UniqueName: \"kubernetes.io/projected/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-kube-api-access-d4tq7\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.585193 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:52.585107 2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found Jun 07 09:15:52.585193 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:52.585188 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs podName:45ea3af8-7dfd-4d91-b6cc-700f89b4494a nodeName:}" failed. No retries permitted until 2026-06-07 09:15:53.085172283 +0000 UTC m=+445.086177384 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs") pod "kube-auth-proxy-f78cc86d8-gzprl" (UID: "45ea3af8-7dfd-4d91-b6cc-700f89b4494a") : secret "kube-auth-proxy-tls" not found Jun 07 09:15:52.587333 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.587303 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tmp\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.594556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.594536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4tq7\" (UniqueName: \"kubernetes.io/projected/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-kube-api-access-d4tq7\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:52.656132 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.656101 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.686331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.686303 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fsdm\" (UniqueName: \"kubernetes.io/projected/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-kube-api-access-5fsdm\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.686431 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.686336 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.756948 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.756914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:52.787229 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.787198 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fsdm\" (UniqueName: \"kubernetes.io/projected/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-kube-api-access-5fsdm\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.787229 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.787233 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.787447 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:52.787431 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 07 09:15:52.787485 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:52.787478 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert podName:5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa nodeName:}" failed. No retries permitted until 2026-06-07 09:15:53.287465327 +0000 UTC m=+445.288470428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert") pod "odh-model-controller-6dc94f689f-hw6j8" (UID: "5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa") : secret "odh-model-controller-webhook-cert" not found Jun 07 09:15:52.796526 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.796496 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fsdm\" (UniqueName: \"kubernetes.io/projected/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-kube-api-access-5fsdm\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:52.856974 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:52.856909 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:53.089712 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.089680 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:53.092165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.092139 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/45ea3af8-7dfd-4d91-b6cc-700f89b4494a-tls-certs\") pod \"kube-auth-proxy-f78cc86d8-gzprl\" (UID: \"45ea3af8-7dfd-4d91-b6cc-700f89b4494a\") " pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:53.133463 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.133395 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ztfgs"] Jun 07 09:15:53.135631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.135612 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.135823 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.135781 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:53.138479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.138460 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jun 07 09:15:53.138479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.138474 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-94xf7\"" Jun 07 09:15:53.146210 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.146190 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ztfgs"] Jun 07 09:15:53.263895 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.263869 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" Jun 07 09:15:53.292037 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.292009 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:53.292165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.292049 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e30405c-5d9b-4455-9104-59def15c6f19-tls-certs\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.292165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.292099 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxj5p\" (UniqueName: \"kubernetes.io/projected/8e30405c-5d9b-4455-9104-59def15c6f19-kube-api-access-lxj5p\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.292242 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:53.292166 2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 07 09:15:53.292242 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:15:53.292228 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert podName:5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa nodeName:}" failed. No retries permitted until 2026-06-07 09:15:54.292214039 +0000 UTC m=+446.293219140 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert") pod "odh-model-controller-6dc94f689f-hw6j8" (UID: "5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa") : secret "odh-model-controller-webhook-cert" not found Jun 07 09:15:53.349166 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.349135 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:53.393524 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.393447 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e30405c-5d9b-4455-9104-59def15c6f19-tls-certs\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.393662 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.393521 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxj5p\" (UniqueName: \"kubernetes.io/projected/8e30405c-5d9b-4455-9104-59def15c6f19-kube-api-access-lxj5p\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.396150 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.396128 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8e30405c-5d9b-4455-9104-59def15c6f19-tls-certs\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.397352 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.397330 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl"] Jun 07 09:15:53.399384 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:53.399360 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ea3af8_7dfd_4d91_b6cc_700f89b4494a.slice/crio-dafd934e070e6ead263b61255e48fd4344b06326480eab5e57e12bbce5b0a05f WatchSource:0}: Error finding container dafd934e070e6ead263b61255e48fd4344b06326480eab5e57e12bbce5b0a05f: Status 404 returned error can't find the container with id dafd934e070e6ead263b61255e48fd4344b06326480eab5e57e12bbce5b0a05f Jun 07 09:15:53.402069 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.402050 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxj5p\" (UniqueName: \"kubernetes.io/projected/8e30405c-5d9b-4455-9104-59def15c6f19-kube-api-access-lxj5p\") pod \"model-serving-api-5dd488fc48-ztfgs\" (UID: \"8e30405c-5d9b-4455-9104-59def15c6f19\") " pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.446552 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.446528 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:53.568981 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.568922 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-5dd488fc48-ztfgs"] Jun 07 09:15:53.571175 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:53.571148 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e30405c_5d9b_4455_9104_59def15c6f19.slice/crio-e1b4dd4c0b87c4fc7ff4835ce4cb1c623d3cd11ecd421318ce2caa4a921416c5 WatchSource:0}: Error finding container e1b4dd4c0b87c4fc7ff4835ce4cb1c623d3cd11ecd421318ce2caa4a921416c5: Status 404 returned error can't find the container with id e1b4dd4c0b87c4fc7ff4835ce4cb1c623d3cd11ecd421318ce2caa4a921416c5 Jun 07 09:15:53.608570 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:53.608542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:15:54.020476 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.020440 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" event={"ID":"8e30405c-5d9b-4455-9104-59def15c6f19","Type":"ContainerStarted","Data":"e1b4dd4c0b87c4fc7ff4835ce4cb1c623d3cd11ecd421318ce2caa4a921416c5"} Jun 07 09:15:54.021714 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.021680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" event={"ID":"45ea3af8-7dfd-4d91-b6cc-700f89b4494a","Type":"ContainerStarted","Data":"dafd934e070e6ead263b61255e48fd4344b06326480eab5e57e12bbce5b0a05f"} Jun 07 09:15:54.301624 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.301543 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:54.307093 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.305470 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa-cert\") pod \"odh-model-controller-6dc94f689f-hw6j8\" (UID: \"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa\") " pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:54.378595 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.378563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:54.554259 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:54.554133 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-6dc94f689f-hw6j8"] Jun 07 09:15:54.558115 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:54.558073 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cf5f7a6_5ebf_4ffa_b692_1774e39dc8fa.slice/crio-d23b9d1cc36811baed7439f0cf8972d44635d0ab07cb615d0a53ffa5263cb078 WatchSource:0}: Error finding container d23b9d1cc36811baed7439f0cf8972d44635d0ab07cb615d0a53ffa5263cb078: Status 404 returned error can't find the container with id d23b9d1cc36811baed7439f0cf8972d44635d0ab07cb615d0a53ffa5263cb078 Jun 07 09:15:55.029837 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.029772 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" event={"ID":"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa","Type":"ContainerStarted","Data":"d23b9d1cc36811baed7439f0cf8972d44635d0ab07cb615d0a53ffa5263cb078"} Jun 07 09:15:55.443144 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.443114 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7"] Jun 07 09:15:55.445404 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.445388 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.448153 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.448126 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:15:55.449612 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.449592 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:15:55.449687 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.449601 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:15:55.453079 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.453057 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7"] Jun 07 09:15:55.512251 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.512213 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.512399 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.512262 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzkxt\" (UniqueName: \"kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.512399 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.512353 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.613095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.613062 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.613260 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.613161 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.613260 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.613195 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzkxt\" (UniqueName: \"kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.613522 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.613501 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.613595 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.613538 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.621932 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.621907 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzkxt\" (UniqueName: \"kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt\") pod \"4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:55.758597 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:55.758510 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:15:56.655693 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:56.655668 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7"] Jun 07 09:15:56.657239 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:15:56.657215 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a5c2bfb_db79_44e9_86ad_c4199db25694.slice/crio-06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2 WatchSource:0}: Error finding container 06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2: Status 404 returned error can't find the container with id 06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2 Jun 07 09:15:57.043729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.043442 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" event={"ID":"8e30405c-5d9b-4455-9104-59def15c6f19","Type":"ContainerStarted","Data":"6bd522f2df60543d6b0cfe56e9e4faa088f0c73294d682e5dc5f1a71f5b824dd"} Jun 07 09:15:57.043729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.043571 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:15:57.045579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.045539 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerID="ce6932186226f2e25d5709cf2d6e7360cddabb959365034d17c1a23dfcf401fb" exitCode=0 Jun 07 09:15:57.045579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.045571 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" event={"ID":"4a5c2bfb-db79-44e9-86ad-c4199db25694","Type":"ContainerDied","Data":"ce6932186226f2e25d5709cf2d6e7360cddabb959365034d17c1a23dfcf401fb"} Jun 07 09:15:57.045579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.045618 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" event={"ID":"4a5c2bfb-db79-44e9-86ad-c4199db25694","Type":"ContainerStarted","Data":"06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2"} Jun 07 09:15:57.047710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.047685 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" event={"ID":"45ea3af8-7dfd-4d91-b6cc-700f89b4494a","Type":"ContainerStarted","Data":"f7b155f672952e98f90cd436702aa833c59fd5b072ba07c235151607f85a0ac4"} Jun 07 09:15:57.062371 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.062327 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" podStartSLOduration=1.116721772 podStartE2EDuration="4.062314613s" totalImagesPullingTime="2.945592841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:57.060476543 +0000 UTC m=+449.061481667" watchObservedRunningTime="2026-06-07 09:15:57.062314613 +0000 UTC m=+449.063319736" Jun 07 09:15:57.099592 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:57.099542 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-f78cc86d8-gzprl" podStartSLOduration=1.969974685 podStartE2EDuration="5.099494509s" totalImagesPullingTime="3.129519824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:57.098440829 +0000 UTC m=+449.099445945" watchObservedRunningTime="2026-06-07 09:15:57.099494509 +0000 UTC m=+449.100499633" Jun 07 09:15:59.063153 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:59.063045 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" event={"ID":"5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa","Type":"ContainerStarted","Data":"00a1bbce0947a8289aab0fb9e865cf7f43576135d74e61276128600e4b8a8eee"} Jun 07 09:15:59.063153 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:59.063127 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:15:59.064846 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:59.064820 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerID="517a8d6c6ff43b67428b77229c562a87808746096b817df95887ee78ceeb43cd" exitCode=0 Jun 07 09:15:59.064957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:59.064860 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" event={"ID":"4a5c2bfb-db79-44e9-86ad-c4199db25694","Type":"ContainerDied","Data":"517a8d6c6ff43b67428b77229c562a87808746096b817df95887ee78ceeb43cd"} Jun 07 09:15:59.086041 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:15:59.085996 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" podStartSLOduration=3.163961415 podStartE2EDuration="7.085980821s" totalImagesPullingTime="3.922019406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:15:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:15:59.084318745 +0000 UTC m=+451.085323869" watchObservedRunningTime="2026-06-07 09:15:59.085980821 +0000 UTC m=+451.086986003" Jun 07 09:16:00.081689 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:00.081651 2562 generic.go:356] "Generic (PLEG): container finished" podID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerID="25f2fe9fdb9846bef119e902ce139ce56046f62ac94b738173866453c9a2ddfc" exitCode=0 Jun 07 09:16:00.082217 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:00.081737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" event={"ID":"4a5c2bfb-db79-44e9-86ad-c4199db25694","Type":"ContainerDied","Data":"25f2fe9fdb9846bef119e902ce139ce56046f62ac94b738173866453c9a2ddfc"} Jun 07 09:16:01.188513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.188476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:01.217602 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.217578 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:16:01.244402 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244374 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fvx8k"] Jun 07 09:16:01.244657 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244646 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="extract" Jun 07 09:16:01.244710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244660 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="extract" Jun 07 09:16:01.244710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244683 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="util" Jun 07 09:16:01.244710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244689 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="util" Jun 07 09:16:01.244710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244696 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="pull" Jun 07 09:16:01.244710 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244701 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="pull" Jun 07 09:16:01.244897 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.244747 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a5c2bfb-db79-44e9-86ad-c4199db25694" containerName="extract" Jun 07 09:16:01.246658 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.246637 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.249550 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.249529 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jun 07 09:16:01.250076 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.250059 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-zwpdc\"" Jun 07 09:16:01.259283 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.259261 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fvx8k"] Jun 07 09:16:01.260685 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.260665 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5"] Jun 07 09:16:01.262580 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.262563 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.265366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.265342 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jun 07 09:16:01.265458 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.265386 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-r84wd\"" Jun 07 09:16:01.274159 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.274137 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5"] Jun 07 09:16:01.290346 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.290312 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:01.363315 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363259 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kzkxt\" (UniqueName: \"kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt\") pod \"4a5c2bfb-db79-44e9-86ad-c4199db25694\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " Jun 07 09:16:01.363315 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363308 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util\") pod \"4a5c2bfb-db79-44e9-86ad-c4199db25694\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " Jun 07 09:16:01.363465 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363360 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle\") pod \"4a5c2bfb-db79-44e9-86ad-c4199db25694\" (UID: \"4a5c2bfb-db79-44e9-86ad-c4199db25694\") " Jun 07 09:16:01.363465 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363430 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c2qdl\" (UniqueName: \"kubernetes.io/projected/c9181244-b67c-4dde-acaf-2f99f0ae96bd-kube-api-access-c2qdl\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.363465 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363455 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9181244-b67c-4dde-acaf-2f99f0ae96bd-cert\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.363625 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363472 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.363625 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.363568 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82zqd\" (UniqueName: \"kubernetes.io/projected/6eb8ea7b-a00d-40bc-9862-1551322add25-kube-api-access-82zqd\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.364271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.364250 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle" (OuterVolumeSpecName: "bundle") pod "4a5c2bfb-db79-44e9-86ad-c4199db25694" (UID: "4a5c2bfb-db79-44e9-86ad-c4199db25694"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:16:01.365416 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.365390 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt" (OuterVolumeSpecName: "kube-api-access-kzkxt") pod "4a5c2bfb-db79-44e9-86ad-c4199db25694" (UID: "4a5c2bfb-db79-44e9-86ad-c4199db25694"). InnerVolumeSpecName "kube-api-access-kzkxt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:16:01.369117 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.369092 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util" (OuterVolumeSpecName: "util") pod "4a5c2bfb-db79-44e9-86ad-c4199db25694" (UID: "4a5c2bfb-db79-44e9-86ad-c4199db25694"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:16:01.464982 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.464955 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82zqd\" (UniqueName: \"kubernetes.io/projected/6eb8ea7b-a00d-40bc-9862-1551322add25-kube-api-access-82zqd\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465006 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c2qdl\" (UniqueName: \"kubernetes.io/projected/c9181244-b67c-4dde-acaf-2f99f0ae96bd-kube-api-access-c2qdl\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465027 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9181244-b67c-4dde-acaf-2f99f0ae96bd-cert\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465076 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465087 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzkxt\" (UniqueName: \"kubernetes.io/projected/4a5c2bfb-db79-44e9-86ad-c4199db25694-kube-api-access-kzkxt\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:01.465135 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.465095 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4a5c2bfb-db79-44e9-86ad-c4199db25694-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:01.465393 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:16:01.465162 2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jun 07 09:16:01.465393 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:16:01.465211 2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert podName:6eb8ea7b-a00d-40bc-9862-1551322add25 nodeName:}" failed. No retries permitted until 2026-06-07 09:16:01.965194843 +0000 UTC m=+453.966199945 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert") pod "llmisvc-controller-manager-864bfd7bc9-cqmv5" (UID: "6eb8ea7b-a00d-40bc-9862-1551322add25") : secret "llmisvc-webhook-server-cert" not found Jun 07 09:16:01.467547 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.467520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c9181244-b67c-4dde-acaf-2f99f0ae96bd-cert\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.489296 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.489273 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-82zqd\" (UniqueName: \"kubernetes.io/projected/6eb8ea7b-a00d-40bc-9862-1551322add25-kube-api-access-82zqd\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.495542 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.495510 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c2qdl\" (UniqueName: \"kubernetes.io/projected/c9181244-b67c-4dde-acaf-2f99f0ae96bd-kube-api-access-c2qdl\") pod \"kserve-controller-manager-69fc866d76-fvx8k\" (UID: \"c9181244-b67c-4dde-acaf-2f99f0ae96bd\") " pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.542974 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.542946 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:01.557004 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.556977 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:01.642474 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.642363 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:01.694300 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.694155 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-69fc866d76-fvx8k"] Jun 07 09:16:01.696396 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:01.696369 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9181244_b67c_4dde_acaf_2f99f0ae96bd.slice/crio-70e1b4766ca9ad9d7106ac58e2134bd57064e2c24c2cb4b832369ddddeca5569 WatchSource:0}: Error finding container 70e1b4766ca9ad9d7106ac58e2134bd57064e2c24c2cb4b832369ddddeca5569: Status 404 returned error can't find the container with id 70e1b4766ca9ad9d7106ac58e2134bd57064e2c24c2cb4b832369ddddeca5569 Jun 07 09:16:01.969109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.969079 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:01.971464 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:01.971437 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6eb8ea7b-a00d-40bc-9862-1551322add25-cert\") pod \"llmisvc-controller-manager-864bfd7bc9-cqmv5\" (UID: \"6eb8ea7b-a00d-40bc-9862-1551322add25\") " pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:02.092439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.092406 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" Jun 07 09:16:02.092587 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.092422 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4258531154569488b4b06856545cf53ed48b82e88ddd64b4ce22efab06vnhh7" event={"ID":"4a5c2bfb-db79-44e9-86ad-c4199db25694","Type":"ContainerDied","Data":"06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2"} Jun 07 09:16:02.092587 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.092539 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06707b7277671a4e57aad53be52f7566d4c671220b52ff51cc0a37a1ab53c7a2" Jun 07 09:16:02.093705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.093680 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" event={"ID":"c9181244-b67c-4dde-acaf-2f99f0ae96bd","Type":"ContainerStarted","Data":"70e1b4766ca9ad9d7106ac58e2134bd57064e2c24c2cb4b832369ddddeca5569"} Jun 07 09:16:02.172712 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.172691 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:02.307840 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.307798 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5"] Jun 07 09:16:02.309506 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:02.309473 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod6eb8ea7b_a00d_40bc_9862_1551322add25.slice/crio-032e9265237e200870b1e8fb120e9f478bf5d2bca3b63e5647fc5a8dc7d2e88e WatchSource:0}: Error finding container 032e9265237e200870b1e8fb120e9f478bf5d2bca3b63e5647fc5a8dc7d2e88e: Status 404 returned error can't find the container with id 032e9265237e200870b1e8fb120e9f478bf5d2bca3b63e5647fc5a8dc7d2e88e Jun 07 09:16:02.493379 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.493280 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:02.598536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:02.598498 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:03.100968 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:03.100932 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" event={"ID":"6eb8ea7b-a00d-40bc-9862-1551322add25","Type":"ContainerStarted","Data":"032e9265237e200870b1e8fb120e9f478bf5d2bca3b63e5647fc5a8dc7d2e88e"} Jun 07 09:16:05.113015 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:05.112975 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" event={"ID":"c9181244-b67c-4dde-acaf-2f99f0ae96bd","Type":"ContainerStarted","Data":"8dc199d1ed97f1716c903d2560e07dfde331985aea33d5985d58db6f406dc2ad"} Jun 07 09:16:05.113471 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:05.113080 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:05.131059 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:05.131011 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" podStartSLOduration=1.380458581 podStartE2EDuration="4.130990531s" totalImagesPullingTime="2.75053195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:16:05.129704999 +0000 UTC m=+457.130710131" watchObservedRunningTime="2026-06-07 09:16:05.130990531 +0000 UTC m=+457.131995658" Jun 07 09:16:06.119887 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:06.119849 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" event={"ID":"6eb8ea7b-a00d-40bc-9862-1551322add25","Type":"ContainerStarted","Data":"5f9df3ec7bb53d6a96357c34c7a91870048445d41d9d3fe00e3801268452c9c9"} Jun 07 09:16:06.120271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:06.119937 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:06.138554 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:06.138510 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" podStartSLOduration=2.017364749 podStartE2EDuration="5.138493459s" totalImagesPullingTime="3.12112871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:16:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:16:06.136405996 +0000 UTC m=+458.137411114" watchObservedRunningTime="2026-06-07 09:16:06.138493459 +0000 UTC m=+458.139498582" Jun 07 09:16:08.061042 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:08.061006 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-5dd488fc48-ztfgs" Jun 07 09:16:09.080972 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:09.080937 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:09.184974 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:09.184943 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:10.083940 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.083905 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-6dc94f689f-hw6j8" Jun 07 09:16:10.236599 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.236546 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:10.389629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.389542 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:10.446987 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.446956 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb"] Jun 07 09:16:10.456509 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.456486 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.461392 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.461367 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:16:10.461605 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.461586 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:16:10.461699 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.461677 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:16:10.469368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.469346 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb"] Jun 07 09:16:10.639733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.639659 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8kbdv\" (UniqueName: \"kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.639733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.639697 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.639942 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.639778 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.741099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.741068 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.741250 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.741118 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8kbdv\" (UniqueName: \"kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.741250 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.741223 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.741481 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.741461 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.741520 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.741498 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.750552 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.750532 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8kbdv\" (UniqueName: \"kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv\") pod \"c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.766491 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.766466 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:10.921600 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.921577 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb"] Jun 07 09:16:10.923257 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:10.923226 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78e3f01a_b2c7_4908_b840_03308a8050f3.slice/crio-b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb WatchSource:0}: Error finding container b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb: Status 404 returned error can't find the container with id b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb Jun 07 09:16:10.994262 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:10.994238 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:11.111023 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.110995 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:11.145815 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.145756 2562 generic.go:356] "Generic (PLEG): container finished" podID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerID="1e725e7ac8748effd8785ebaa64a0437fb6ee76124cb02b70b3f825fc8d402bb" exitCode=0 Jun 07 09:16:11.145927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.145845 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" event={"ID":"78e3f01a-b2c7-4908-b840-03308a8050f3","Type":"ContainerDied","Data":"1e725e7ac8748effd8785ebaa64a0437fb6ee76124cb02b70b3f825fc8d402bb"} Jun 07 09:16:11.145927 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.145901 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" event={"ID":"78e3f01a-b2c7-4908-b840-03308a8050f3","Type":"ContainerStarted","Data":"b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb"} Jun 07 09:16:11.593454 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.593424 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:11.750491 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.750459 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-87pfd"] Jun 07 09:16:11.753719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.753698 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.756995 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.756948 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"kube-root-ca.crt\"" Jun 07 09:16:11.757096 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.757055 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-operators\"/\"openshift-service-ca.crt\"" Jun 07 09:16:11.757196 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.757163 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-operators\"/\"servicemesh-operator3-dockercfg-dkzs8\"" Jun 07 09:16:11.769020 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.768998 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-87pfd"] Jun 07 09:16:11.848969 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.848913 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gs8v\" (UniqueName: \"kubernetes.io/projected/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-kube-api-access-4gs8v\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.849068 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.848994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-operator-config\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.950006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.949981 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-operator-config\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.950104 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.950072 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gs8v\" (UniqueName: \"kubernetes.io/projected/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-kube-api-access-4gs8v\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.952241 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.952220 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"operator-config\" (UniqueName: \"kubernetes.io/downward-api/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-operator-config\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:11.959483 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:11.959460 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gs8v\" (UniqueName: \"kubernetes.io/projected/4546ac4d-4d84-4794-a7b2-a8d1c3b74066-kube-api-access-4gs8v\") pod \"servicemesh-operator3-57f65f65fb-87pfd\" (UID: \"4546ac4d-4d84-4794-a7b2-a8d1c3b74066\") " pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:12.067024 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:12.066994 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:12.154396 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:12.154361 2562 generic.go:356] "Generic (PLEG): container finished" podID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerID="fedb4d34435d5b36894b34cad108adbaf60cde9b7995524d09c055d826480b2a" exitCode=0 Jun 07 09:16:12.154742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:12.154407 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" event={"ID":"78e3f01a-b2c7-4908-b840-03308a8050f3","Type":"ContainerDied","Data":"fedb4d34435d5b36894b34cad108adbaf60cde9b7995524d09c055d826480b2a"} Jun 07 09:16:12.219111 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:12.219089 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-operators/servicemesh-operator3-57f65f65fb-87pfd"] Jun 07 09:16:12.220608 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:12.220588 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4546ac4d_4d84_4794_a7b2_a8d1c3b74066.slice/crio-66d85399d98a6d89b9076005b3f1a2f2d699df304381064d1ef427227b34aff0 WatchSource:0}: Error finding container 66d85399d98a6d89b9076005b3f1a2f2d699df304381064d1ef427227b34aff0: Status 404 returned error can't find the container with id 66d85399d98a6d89b9076005b3f1a2f2d699df304381064d1ef427227b34aff0 Jun 07 09:16:13.170887 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:13.170842 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" event={"ID":"4546ac4d-4d84-4794-a7b2-a8d1c3b74066","Type":"ContainerStarted","Data":"66d85399d98a6d89b9076005b3f1a2f2d699df304381064d1ef427227b34aff0"} Jun 07 09:16:13.173822 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:13.173770 2562 generic.go:356] "Generic (PLEG): container finished" podID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerID="c0f1d3410256d4aec23d4d6fd25c7aec0a45da67cf81474ec193905b12b76087" exitCode=0 Jun 07 09:16:13.173968 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:13.173884 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" event={"ID":"78e3f01a-b2c7-4908-b840-03308a8050f3","Type":"ContainerDied","Data":"c0f1d3410256d4aec23d4d6fd25c7aec0a45da67cf81474ec193905b12b76087"} Jun 07 09:16:14.572871 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.572847 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:14.672517 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.672497 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle\") pod \"78e3f01a-b2c7-4908-b840-03308a8050f3\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " Jun 07 09:16:14.672628 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.672550 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util\") pod \"78e3f01a-b2c7-4908-b840-03308a8050f3\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " Jun 07 09:16:14.672628 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.672572 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8kbdv\" (UniqueName: \"kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv\") pod \"78e3f01a-b2c7-4908-b840-03308a8050f3\" (UID: \"78e3f01a-b2c7-4908-b840-03308a8050f3\") " Jun 07 09:16:14.673481 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.673455 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle" (OuterVolumeSpecName: "bundle") pod "78e3f01a-b2c7-4908-b840-03308a8050f3" (UID: "78e3f01a-b2c7-4908-b840-03308a8050f3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:16:14.674917 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.674890 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv" (OuterVolumeSpecName: "kube-api-access-8kbdv") pod "78e3f01a-b2c7-4908-b840-03308a8050f3" (UID: "78e3f01a-b2c7-4908-b840-03308a8050f3"). InnerVolumeSpecName "kube-api-access-8kbdv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:16:14.678036 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.678013 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util" (OuterVolumeSpecName: "util") pod "78e3f01a-b2c7-4908-b840-03308a8050f3" (UID: "78e3f01a-b2c7-4908-b840-03308a8050f3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:16:14.773769 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.773729 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:14.773769 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.773768 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78e3f01a-b2c7-4908-b840-03308a8050f3-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:14.774029 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:14.773782 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8kbdv\" (UniqueName: \"kubernetes.io/projected/78e3f01a-b2c7-4908-b840-03308a8050f3-kube-api-access-8kbdv\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:16:15.186305 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.186276 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" Jun 07 09:16:15.186484 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.186273 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/c34dbd22d61532abdd91a8be490d70c9b1095425e8ccf73b542d2e762acwtqb" event={"ID":"78e3f01a-b2c7-4908-b840-03308a8050f3","Type":"ContainerDied","Data":"b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb"} Jun 07 09:16:15.186484 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.186404 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b4a46d53cea2b7d4368c379c61d5b104d9f2b360c82e126c3f133f63c74660bb" Jun 07 09:16:15.188541 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.188487 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" event={"ID":"4546ac4d-4d84-4794-a7b2-a8d1c3b74066","Type":"ContainerStarted","Data":"888c84a9a8e624f77c032e4c5dbec72b57e31e0f6d10dda789395dd54a3843ac"} Jun 07 09:16:15.188673 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.188618 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:15.210040 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:15.209997 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" podStartSLOduration=1.817896373 podStartE2EDuration="4.209986141s" totalImagesPullingTime="2.392089768s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:16:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:16:15.207501506 +0000 UTC m=+467.208506628" watchObservedRunningTime="2026-06-07 09:16:15.209986141 +0000 UTC m=+467.210991264" Jun 07 09:16:16.278189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:16.278154 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:19.038339 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:19.038295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:20.771146 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:20.771085 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:20.872005 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:20.871934 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.053685 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.053604 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.643089 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643056 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr"] Jun 07 09:16:21.643534 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643515 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="extract" Jun 07 09:16:21.643579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643540 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="extract" Jun 07 09:16:21.643579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643566 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="util" Jun 07 09:16:21.643579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643575 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="util" Jun 07 09:16:21.643674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643594 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="pull" Jun 07 09:16:21.643674 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643604 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="pull" Jun 07 09:16:21.643733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.643687 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="78e3f01a-b2c7-4908-b840-03308a8050f3" containerName="extract" Jun 07 09:16:21.647077 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.647056 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.650112 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.650086 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jun 07 09:16:21.650232 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.650125 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-8tw95\"" Jun 07 09:16:21.650232 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.650127 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jun 07 09:16:21.650232 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.650207 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 07 09:16:21.650403 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.650388 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jun 07 09:16:21.651905 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.651880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.658638 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.658618 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr"] Jun 07 09:16:21.751740 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.751709 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.822795 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822750 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61902ac-e5ce-4728-bde1-3388deca1c93-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.822795 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822785 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-js2bk\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-kube-api-access-js2bk\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.823439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822829 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.823439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822898 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.823439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822930 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.823439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.822980 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.823439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.823026 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.852738 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.852702 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.904799 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.904718 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:21.923857 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.923823 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.923866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.923889 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.923935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924006 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.923989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.924044 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61902ac-e5ce-4728-bde1-3388deca1c93-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.924069 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-js2bk\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-kube-api-access-js2bk\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.924689 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.924662 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.926386 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.926363 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-cacerts\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.926501 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.926422 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/a61902ac-e5ce-4728-bde1-3388deca1c93-local-certs\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.926501 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.926490 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.926599 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.926536 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-kubeconfig\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.932952 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.932925 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-js2bk\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-kube-api-access-js2bk\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.933054 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.933036 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a61902ac-e5ce-4728-bde1-3388deca1c93-istio-token\") pod \"istiod-openshift-gateway-75c67f8887-ms5jr\" (UID: \"a61902ac-e5ce-4728-bde1-3388deca1c93\") " pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:21.957957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:21.957933 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:22.095448 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:22.095417 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr"] Jun 07 09:16:22.097093 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:22.097065 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda61902ac_e5ce_4728_bde1_3388deca1c93.slice/crio-2d7690f6c2a7ea8ba37a2bb493db1f566089094e9fd8e649d14bd94981df9c22 WatchSource:0}: Error finding container 2d7690f6c2a7ea8ba37a2bb493db1f566089094e9fd8e649d14bd94981df9c22: Status 404 returned error can't find the container with id 2d7690f6c2a7ea8ba37a2bb493db1f566089094e9fd8e649d14bd94981df9c22 Jun 07 09:16:22.198672 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:22.198638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:22.223408 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:22.223376 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" event={"ID":"a61902ac-e5ce-4728-bde1-3388deca1c93","Type":"ContainerStarted","Data":"2d7690f6c2a7ea8ba37a2bb493db1f566089094e9fd8e649d14bd94981df9c22"} Jun 07 09:16:24.607290 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:24.607244 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:16:24.607558 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:24.607320 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:16:25.256763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.256107 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" event={"ID":"a61902ac-e5ce-4728-bde1-3388deca1c93","Type":"ContainerStarted","Data":"400906d0f9610650e653e9ed5da6935f9e7730631c67e7b1f2d07a5168b18021"} Jun 07 09:16:25.256763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.256649 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:25.258589 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.258546 2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-75c67f8887-ms5jr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jun 07 09:16:25.258941 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.258836 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" podUID="a61902ac-e5ce-4728-bde1-3388deca1c93" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 07 09:16:25.280471 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.280412 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" podStartSLOduration=1.7725987829999998 podStartE2EDuration="4.280391668s" totalImagesPullingTime="2.507792885s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:16:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:16:25.278001542 +0000 UTC m=+477.279006677" watchObservedRunningTime="2026-06-07 09:16:25.280391668 +0000 UTC m=+477.281396791" Jun 07 09:16:25.292252 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.292218 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:25.391562 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.391525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:25.584925 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.584844 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:25.772843 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.772790 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:25.885860 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:25.885745 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:26.196055 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:26.196022 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-operators/servicemesh-operator3-57f65f65fb-87pfd" Jun 07 09:16:26.262887 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:26.262862 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-75c67f8887-ms5jr" Jun 07 09:16:29.424706 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.424669 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-8f57c774b-vrg57"] Jun 07 09:16:29.446994 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.446961 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:29.488180 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.488156 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-8f57c774b-vrg57"] Jun 07 09:16:29.488296 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.488236 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.491218 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.491198 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 07 09:16:29.491329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.491269 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7hqtt\"" Jun 07 09:16:29.588942 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.588909 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5a6ba7bb-ba43-4924-9b65-2347627cda5a-redirect-config\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.589075 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.588966 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fggwk\" (UniqueName: \"kubernetes.io/projected/5a6ba7bb-ba43-4924-9b65-2347627cda5a-kube-api-access-fggwk\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.689979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.689911 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fggwk\" (UniqueName: \"kubernetes.io/projected/5a6ba7bb-ba43-4924-9b65-2347627cda5a-kube-api-access-fggwk\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.689979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.689977 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5a6ba7bb-ba43-4924-9b65-2347627cda5a-redirect-config\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.690615 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.690592 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/5a6ba7bb-ba43-4924-9b65-2347627cda5a-redirect-config\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.698294 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.698264 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fggwk\" (UniqueName: \"kubernetes.io/projected/5a6ba7bb-ba43-4924-9b65-2347627cda5a-kube-api-access-fggwk\") pod \"dashboard-redirect-8f57c774b-vrg57\" (UID: \"5a6ba7bb-ba43-4924-9b65-2347627cda5a\") " pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.797534 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.797504 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" Jun 07 09:16:29.958464 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:29.958435 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-8f57c774b-vrg57"] Jun 07 09:16:29.960171 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:16:29.960140 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6ba7bb_ba43_4924_9b65_2347627cda5a.slice/crio-1b41e4ca7ab741a2502f73b49574cb967307a072dadc94e85a15d465163ed711 WatchSource:0}: Error finding container 1b41e4ca7ab741a2502f73b49574cb967307a072dadc94e85a15d465163ed711: Status 404 returned error can't find the container with id 1b41e4ca7ab741a2502f73b49574cb967307a072dadc94e85a15d465163ed711 Jun 07 09:16:30.288449 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:30.288365 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" event={"ID":"5a6ba7bb-ba43-4924-9b65-2347627cda5a","Type":"ContainerStarted","Data":"1b41e4ca7ab741a2502f73b49574cb967307a072dadc94e85a15d465163ed711"} Jun 07 09:16:30.478760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:30.478728 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:32.302106 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:32.302072 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" event={"ID":"5a6ba7bb-ba43-4924-9b65-2347627cda5a","Type":"ContainerStarted","Data":"4a3c986d3d82e228edc9b3115a1ed8b7ebf152157ec816c15b316f6c9973deda"} Jun 07 09:16:32.317824 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:32.317749 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-8f57c774b-vrg57" podStartSLOduration=1.525555045 podStartE2EDuration="3.317733224s" totalImagesPullingTime="1.792178179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:16:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:16:32.317102893 +0000 UTC m=+484.318108016" watchObservedRunningTime="2026-06-07 09:16:32.317733224 +0000 UTC m=+484.318738355" Jun 07 09:16:36.125748 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:36.125720 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-69fc866d76-fvx8k" Jun 07 09:16:37.126526 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:37.126499 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-864bfd7bc9-cqmv5" Jun 07 09:16:53.196930 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:53.196892 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:16:54.185858 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:16:54.185801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:06.690452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:06.690420 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:06.793652 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:06.793618 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:08.496080 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.496010 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64"] Jun 07 09:17:08.499385 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.499366 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.502437 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.502417 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 07 09:17:08.502533 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.502425 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 07 09:17:08.504254 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.504235 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jdchw\"" Jun 07 09:17:08.515387 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.512261 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64"] Jun 07 09:17:08.604766 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.604717 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r222s\" (UniqueName: \"kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.604942 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.604781 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.604942 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.604866 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.705210 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.705181 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r222s\" (UniqueName: \"kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.705339 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.705222 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.705339 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.705246 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.705566 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.705547 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.705608 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.705581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.714475 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.714447 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r222s\" (UniqueName: \"kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.817238 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.817178 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:08.946491 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:08.946466 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64"] Jun 07 09:17:08.948194 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:17:08.948161 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc5f3dba_ac38_4c0f_b616_e523792285cb.slice/crio-54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e WatchSource:0}: Error finding container 54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e: Status 404 returned error can't find the container with id 54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e Jun 07 09:17:09.099093 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.099039 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj"] Jun 07 09:17:09.102433 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.102417 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.110437 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.110413 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj"] Jun 07 09:17:09.209048 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.209019 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.209185 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.209062 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kw7j6\" (UniqueName: \"kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.209185 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.209128 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.310311 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.310287 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.310437 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.310321 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kw7j6\" (UniqueName: \"kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.310437 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.310344 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.310669 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.310652 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.310783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.310766 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.319046 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.319018 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kw7j6\" (UniqueName: \"kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.411454 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.411387 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:09.486202 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.486174 2562 generic.go:356] "Generic (PLEG): container finished" podID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerID="6e484484d599e459c95144dc2b4014771b241f7c4b556b53896d1bec084b4e68" exitCode=0 Jun 07 09:17:09.486350 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.486261 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" event={"ID":"bc5f3dba-ac38-4c0f-b616-e523792285cb","Type":"ContainerDied","Data":"6e484484d599e459c95144dc2b4014771b241f7c4b556b53896d1bec084b4e68"} Jun 07 09:17:09.486350 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.486315 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" event={"ID":"bc5f3dba-ac38-4c0f-b616-e523792285cb","Type":"ContainerStarted","Data":"54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e"} Jun 07 09:17:09.543190 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.543169 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj"] Jun 07 09:17:09.545262 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:17:09.545236 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12deba9f_2cc7_4c42_94bb_fa3d39d72895.slice/crio-91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12 WatchSource:0}: Error finding container 91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12: Status 404 returned error can't find the container with id 91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12 Jun 07 09:17:09.664135 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:17:09.664110 2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12deba9f_2cc7_4c42_94bb_fa3d39d72895.slice/crio-conmon-bfe64847880102e6e220f903eed87626ccb3e78abb08906ba0c968fab3cc18a2.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12deba9f_2cc7_4c42_94bb_fa3d39d72895.slice/crio-bfe64847880102e6e220f903eed87626ccb3e78abb08906ba0c968fab3cc18a2.scope\": RecentStats: unable to find data in memory cache]" Jun 07 09:17:09.699476 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.699453 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc"] Jun 07 09:17:09.702719 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.702705 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.711129 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.711105 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc"] Jun 07 09:17:09.814914 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.814869 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8m759\" (UniqueName: \"kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.815083 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.814926 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.815083 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.814986 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.916309 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.916239 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8m759\" (UniqueName: \"kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.916309 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.916276 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.916446 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.916325 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.916750 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.916728 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.916787 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.916759 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:09.925601 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:09.925581 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8m759\" (UniqueName: \"kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:10.013218 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.013194 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:10.099634 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.099587 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l"] Jun 07 09:17:10.103897 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.103873 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.109769 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.109740 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l"] Jun 07 09:17:10.151334 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.151307 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc"] Jun 07 09:17:10.152034 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:17:10.152013 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod363286c0_98be_4562_bed3_799d4b4d66fd.slice/crio-d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5 WatchSource:0}: Error finding container d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5: Status 404 returned error can't find the container with id d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5 Jun 07 09:17:10.219901 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.219876 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.219993 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.219912 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.220044 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.219969 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25gwv\" (UniqueName: \"kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.320832 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.320777 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.320956 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.320851 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.320956 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.320894 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25gwv\" (UniqueName: \"kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.321103 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.321084 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.321199 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.321181 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.328542 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.328523 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25gwv\" (UniqueName: \"kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.419618 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.419568 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:10.493249 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.493205 2562 generic.go:356] "Generic (PLEG): container finished" podID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerID="1bbdc8cf5b706998f915189139a5edd8a940e26ee83efbe6eda39697e46b1cef" exitCode=0 Jun 07 09:17:10.493411 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.493291 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" event={"ID":"bc5f3dba-ac38-4c0f-b616-e523792285cb","Type":"ContainerDied","Data":"1bbdc8cf5b706998f915189139a5edd8a940e26ee83efbe6eda39697e46b1cef"} Jun 07 09:17:10.495166 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.495145 2562 generic.go:356] "Generic (PLEG): container finished" podID="363286c0-98be-4562-bed3-799d4b4d66fd" containerID="f746ddc791c32b4f4f90b0b36af5b69f1617714cba91ece7324349eb12abfa23" exitCode=0 Jun 07 09:17:10.495261 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.495183 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" event={"ID":"363286c0-98be-4562-bed3-799d4b4d66fd","Type":"ContainerDied","Data":"f746ddc791c32b4f4f90b0b36af5b69f1617714cba91ece7324349eb12abfa23"} Jun 07 09:17:10.495261 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.495239 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" event={"ID":"363286c0-98be-4562-bed3-799d4b4d66fd","Type":"ContainerStarted","Data":"d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5"} Jun 07 09:17:10.496883 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.496859 2562 generic.go:356] "Generic (PLEG): container finished" podID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerID="bfe64847880102e6e220f903eed87626ccb3e78abb08906ba0c968fab3cc18a2" exitCode=0 Jun 07 09:17:10.497019 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.496918 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" event={"ID":"12deba9f-2cc7-4c42-94bb-fa3d39d72895","Type":"ContainerDied","Data":"bfe64847880102e6e220f903eed87626ccb3e78abb08906ba0c968fab3cc18a2"} Jun 07 09:17:10.497019 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.496963 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" event={"ID":"12deba9f-2cc7-4c42-94bb-fa3d39d72895","Type":"ContainerStarted","Data":"91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12"} Jun 07 09:17:10.559659 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:10.558728 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l"] Jun 07 09:17:11.503975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.503899 2562 generic.go:356] "Generic (PLEG): container finished" podID="363286c0-98be-4562-bed3-799d4b4d66fd" containerID="a25cffea80b1ababdb0b7d009fd8ed6544cb83c7fe47e61d20f49594dfb19060" exitCode=0 Jun 07 09:17:11.504095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.503983 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" event={"ID":"363286c0-98be-4562-bed3-799d4b4d66fd","Type":"ContainerDied","Data":"a25cffea80b1ababdb0b7d009fd8ed6544cb83c7fe47e61d20f49594dfb19060"} Jun 07 09:17:11.505793 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.505757 2562 generic.go:356] "Generic (PLEG): container finished" podID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerID="331417babab406769862333388eda00e885b74bbb4d03319dd8c936ff979af9e" exitCode=0 Jun 07 09:17:11.505929 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.505834 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" event={"ID":"12deba9f-2cc7-4c42-94bb-fa3d39d72895","Type":"ContainerDied","Data":"331417babab406769862333388eda00e885b74bbb4d03319dd8c936ff979af9e"} Jun 07 09:17:11.508081 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.508062 2562 generic.go:356] "Generic (PLEG): container finished" podID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerID="10196ffb1b382bd1ef93038116dc7a15b49079f4179967dfb20032a1e33912c1" exitCode=0 Jun 07 09:17:11.508213 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.508113 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" event={"ID":"bc5f3dba-ac38-4c0f-b616-e523792285cb","Type":"ContainerDied","Data":"10196ffb1b382bd1ef93038116dc7a15b49079f4179967dfb20032a1e33912c1"} Jun 07 09:17:11.509660 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.509642 2562 generic.go:356] "Generic (PLEG): container finished" podID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerID="a4ebda2ab7bd92d38fc354dbdc96831e8c8a27711ff710f653b52f276bd7c874" exitCode=0 Jun 07 09:17:11.509747 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.509709 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" event={"ID":"06c188ec-9c16-4250-ba2b-ee5eb7ae3579","Type":"ContainerDied","Data":"a4ebda2ab7bd92d38fc354dbdc96831e8c8a27711ff710f653b52f276bd7c874"} Jun 07 09:17:11.509747 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:11.509744 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" event={"ID":"06c188ec-9c16-4250-ba2b-ee5eb7ae3579","Type":"ContainerStarted","Data":"1b51a3e942b5a7a8bfa5bc3d2afaac4cfc8b94940769aa6cb835417b532d962e"} Jun 07 09:17:12.516163 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.516134 2562 generic.go:356] "Generic (PLEG): container finished" podID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerID="ca02cf83a7cb62afa71a499a85acaf2d2c33b156b07f1e174421ed653715f96a" exitCode=0 Jun 07 09:17:12.516502 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.516212 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" event={"ID":"06c188ec-9c16-4250-ba2b-ee5eb7ae3579","Type":"ContainerDied","Data":"ca02cf83a7cb62afa71a499a85acaf2d2c33b156b07f1e174421ed653715f96a"} Jun 07 09:17:12.518481 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.518458 2562 generic.go:356] "Generic (PLEG): container finished" podID="363286c0-98be-4562-bed3-799d4b4d66fd" containerID="895ca32dd94ccb0a91802dde45c57fb82733c66be6dac76162da56fec428ab47" exitCode=0 Jun 07 09:17:12.518577 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.518535 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" event={"ID":"363286c0-98be-4562-bed3-799d4b4d66fd","Type":"ContainerDied","Data":"895ca32dd94ccb0a91802dde45c57fb82733c66be6dac76162da56fec428ab47"} Jun 07 09:17:12.520605 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.520587 2562 generic.go:356] "Generic (PLEG): container finished" podID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerID="f0d28c6445db97208c8d720449605cee1e6eb1c14b4d7616018e12021ec20967" exitCode=0 Jun 07 09:17:12.520684 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.520650 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" event={"ID":"12deba9f-2cc7-4c42-94bb-fa3d39d72895","Type":"ContainerDied","Data":"f0d28c6445db97208c8d720449605cee1e6eb1c14b4d7616018e12021ec20967"} Jun 07 09:17:12.650839 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.650814 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:12.743572 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.743547 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r222s\" (UniqueName: \"kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s\") pod \"bc5f3dba-ac38-4c0f-b616-e523792285cb\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " Jun 07 09:17:12.743691 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.743678 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle\") pod \"bc5f3dba-ac38-4c0f-b616-e523792285cb\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " Jun 07 09:17:12.743733 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.743706 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util\") pod \"bc5f3dba-ac38-4c0f-b616-e523792285cb\" (UID: \"bc5f3dba-ac38-4c0f-b616-e523792285cb\") " Jun 07 09:17:12.744296 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.744267 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle" (OuterVolumeSpecName: "bundle") pod "bc5f3dba-ac38-4c0f-b616-e523792285cb" (UID: "bc5f3dba-ac38-4c0f-b616-e523792285cb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:12.745656 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.745627 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s" (OuterVolumeSpecName: "kube-api-access-r222s") pod "bc5f3dba-ac38-4c0f-b616-e523792285cb" (UID: "bc5f3dba-ac38-4c0f-b616-e523792285cb"). InnerVolumeSpecName "kube-api-access-r222s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:12.749503 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.749482 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util" (OuterVolumeSpecName: "util") pod "bc5f3dba-ac38-4c0f-b616-e523792285cb" (UID: "bc5f3dba-ac38-4c0f-b616-e523792285cb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:12.845368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.845272 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:12.845368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.845313 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/bc5f3dba-ac38-4c0f-b616-e523792285cb-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:12.845368 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:12.845327 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r222s\" (UniqueName: \"kubernetes.io/projected/bc5f3dba-ac38-4c0f-b616-e523792285cb-kube-api-access-r222s\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.528055 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.528028 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" Jun 07 09:17:13.528556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.528027 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64" event={"ID":"bc5f3dba-ac38-4c0f-b616-e523792285cb","Type":"ContainerDied","Data":"54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e"} Jun 07 09:17:13.528556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.528161 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="54b0aba1fdd466c256f64b7b07a09a7fadce64c4b5ef9575d5c2b792e53c0b0e" Jun 07 09:17:13.530176 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.530149 2562 generic.go:356] "Generic (PLEG): container finished" podID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerID="b832c6b23e168202b2180e20d0bcf27ce08b56eb919a1d472552431a4b3b581c" exitCode=0 Jun 07 09:17:13.530333 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.530237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" event={"ID":"06c188ec-9c16-4250-ba2b-ee5eb7ae3579","Type":"ContainerDied","Data":"b832c6b23e168202b2180e20d0bcf27ce08b56eb919a1d472552431a4b3b581c"} Jun 07 09:17:13.676094 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.676073 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:13.696969 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.696944 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:13.854100 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854017 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kw7j6\" (UniqueName: \"kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6\") pod \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " Jun 07 09:17:13.854100 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854051 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle\") pod \"363286c0-98be-4562-bed3-799d4b4d66fd\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " Jun 07 09:17:13.854100 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854072 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle\") pod \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " Jun 07 09:17:13.854374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854119 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8m759\" (UniqueName: \"kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759\") pod \"363286c0-98be-4562-bed3-799d4b4d66fd\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " Jun 07 09:17:13.854374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854163 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util\") pod \"363286c0-98be-4562-bed3-799d4b4d66fd\" (UID: \"363286c0-98be-4562-bed3-799d4b4d66fd\") " Jun 07 09:17:13.854374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854186 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util\") pod \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\" (UID: \"12deba9f-2cc7-4c42-94bb-fa3d39d72895\") " Jun 07 09:17:13.854573 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854532 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle" (OuterVolumeSpecName: "bundle") pod "363286c0-98be-4562-bed3-799d4b4d66fd" (UID: "363286c0-98be-4562-bed3-799d4b4d66fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:13.854690 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.854661 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle" (OuterVolumeSpecName: "bundle") pod "12deba9f-2cc7-4c42-94bb-fa3d39d72895" (UID: "12deba9f-2cc7-4c42-94bb-fa3d39d72895"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:13.856415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.856390 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6" (OuterVolumeSpecName: "kube-api-access-kw7j6") pod "12deba9f-2cc7-4c42-94bb-fa3d39d72895" (UID: "12deba9f-2cc7-4c42-94bb-fa3d39d72895"). InnerVolumeSpecName "kube-api-access-kw7j6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:13.856518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.856410 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759" (OuterVolumeSpecName: "kube-api-access-8m759") pod "363286c0-98be-4562-bed3-799d4b4d66fd" (UID: "363286c0-98be-4562-bed3-799d4b4d66fd"). InnerVolumeSpecName "kube-api-access-8m759". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:13.859994 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.859974 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util" (OuterVolumeSpecName: "util") pod "363286c0-98be-4562-bed3-799d4b4d66fd" (UID: "363286c0-98be-4562-bed3-799d4b4d66fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:13.862417 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.862390 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util" (OuterVolumeSpecName: "util") pod "12deba9f-2cc7-4c42-94bb-fa3d39d72895" (UID: "12deba9f-2cc7-4c42-94bb-fa3d39d72895"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:13.954955 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.954931 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.954955 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.954953 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.955090 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.954963 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kw7j6\" (UniqueName: \"kubernetes.io/projected/12deba9f-2cc7-4c42-94bb-fa3d39d72895-kube-api-access-kw7j6\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.955090 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.954973 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/363286c0-98be-4562-bed3-799d4b4d66fd-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.955090 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.954999 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/12deba9f-2cc7-4c42-94bb-fa3d39d72895-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:13.955090 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:13.955007 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8m759\" (UniqueName: \"kubernetes.io/projected/363286c0-98be-4562-bed3-799d4b4d66fd-kube-api-access-8m759\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:14.542210 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.542182 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" Jun 07 09:17:14.542210 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.542188 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc" event={"ID":"363286c0-98be-4562-bed3-799d4b4d66fd","Type":"ContainerDied","Data":"d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5"} Jun 07 09:17:14.542711 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.542239 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d3944a9ecfd0f27aedf28ef67699ab6b4c6d5bbc3ef3e7d23c6015b0ba0daad5" Jun 07 09:17:14.544381 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.544351 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" Jun 07 09:17:14.544381 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.544350 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj" event={"ID":"12deba9f-2cc7-4c42-94bb-fa3d39d72895","Type":"ContainerDied","Data":"91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12"} Jun 07 09:17:14.544575 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.544409 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91dc87d731b2721b222d686e38dc59ef2ac929937652692399234317838e1a12" Jun 07 09:17:14.673645 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.673591 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:14.763076 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.763050 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle\") pod \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " Jun 07 09:17:14.763201 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.763158 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util\") pod \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " Jun 07 09:17:14.763201 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.763187 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-25gwv\" (UniqueName: \"kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv\") pod \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\" (UID: \"06c188ec-9c16-4250-ba2b-ee5eb7ae3579\") " Jun 07 09:17:14.763599 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.763573 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle" (OuterVolumeSpecName: "bundle") pod "06c188ec-9c16-4250-ba2b-ee5eb7ae3579" (UID: "06c188ec-9c16-4250-ba2b-ee5eb7ae3579"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:14.765267 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.765244 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv" (OuterVolumeSpecName: "kube-api-access-25gwv") pod "06c188ec-9c16-4250-ba2b-ee5eb7ae3579" (UID: "06c188ec-9c16-4250-ba2b-ee5eb7ae3579"). InnerVolumeSpecName "kube-api-access-25gwv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:14.771062 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.771023 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util" (OuterVolumeSpecName: "util") pod "06c188ec-9c16-4250-ba2b-ee5eb7ae3579" (UID: "06c188ec-9c16-4250-ba2b-ee5eb7ae3579"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:17:14.864227 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.864200 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-25gwv\" (UniqueName: \"kubernetes.io/projected/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-kube-api-access-25gwv\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:14.864227 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.864222 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:14.864227 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:14.864232 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/06c188ec-9c16-4250-ba2b-ee5eb7ae3579-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:15.552120 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:15.552090 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" event={"ID":"06c188ec-9c16-4250-ba2b-ee5eb7ae3579","Type":"ContainerDied","Data":"1b51a3e942b5a7a8bfa5bc3d2afaac4cfc8b94940769aa6cb835417b532d962e"} Jun 07 09:17:15.552120 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:15.552103 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l" Jun 07 09:17:15.552648 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:15.552134 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1b51a3e942b5a7a8bfa5bc3d2afaac4cfc8b94940769aa6cb835417b532d962e" Jun 07 09:17:15.568905 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:17:15.568881 2562 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06c188ec_9c16_4250_ba2b_ee5eb7ae3579.slice/crio-1b51a3e942b5a7a8bfa5bc3d2afaac4cfc8b94940769aa6cb835417b532d962e\": RecentStats: unable to find data in memory cache]" Jun 07 09:17:21.230952 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:21.230888 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:21.320652 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:21.320608 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:21.443919 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:21.443866 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:22.038070 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:22.038038 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:23.076099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:23.076051 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:23.140960 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:23.140898 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:25.771244 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:25.771203 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:25.872293 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:25.872257 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:25.977188 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:25.977151 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:30.068219 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:30.068186 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:32.379207 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:32.379172 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:17:32.503976 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:32.503944 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:32.603354 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:32.603325 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:32.704371 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:32.704340 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:33.679696 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.679666 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p"] Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680003 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="pull" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680015 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="pull" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680027 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="extract" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680033 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="extract" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680041 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="util" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680047 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="util" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680053 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="util" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680076 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="util" Jun 07 09:17:33.680087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680087 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680093 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680102 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="util" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680107 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="util" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680114 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680119 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680126 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="util" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680131 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="util" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680137 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680141 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680148 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680153 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680160 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680165 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="pull" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680171 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680176 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680222 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="363286c0-98be-4562-bed3-799d4b4d66fd" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680231 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="06c188ec-9c16-4250-ba2b-ee5eb7ae3579" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680238 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="12deba9f-2cc7-4c42-94bb-fa3d39d72895" containerName="extract" Jun 07 09:17:33.680366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.680244 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bc5f3dba-ac38-4c0f-b616-e523792285cb" containerName="extract" Jun 07 09:17:33.684556 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.684540 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:33.689254 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.689227 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 07 09:17:33.689254 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.689234 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-kw7sj\"" Jun 07 09:17:33.689430 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.689234 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 07 09:17:33.690720 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.690700 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 07 09:17:33.707980 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.707950 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ms25\" (UniqueName: \"kubernetes.io/projected/3e454ae3-ddd2-49e4-98cd-b409da965ef4-kube-api-access-4ms25\") pod \"dns-operator-controller-manager-864dd78c4c-dvk8p\" (UID: \"3e454ae3-ddd2-49e4-98cd-b409da965ef4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:33.708096 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.708063 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p"] Jun 07 09:17:33.809387 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.809357 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ms25\" (UniqueName: \"kubernetes.io/projected/3e454ae3-ddd2-49e4-98cd-b409da965ef4-kube-api-access-4ms25\") pod \"dns-operator-controller-manager-864dd78c4c-dvk8p\" (UID: \"3e454ae3-ddd2-49e4-98cd-b409da965ef4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:33.818541 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.818520 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ms25\" (UniqueName: \"kubernetes.io/projected/3e454ae3-ddd2-49e4-98cd-b409da965ef4-kube-api-access-4ms25\") pod \"dns-operator-controller-manager-864dd78c4c-dvk8p\" (UID: \"3e454ae3-ddd2-49e4-98cd-b409da965ef4\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:33.994564 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:33.994488 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:34.122907 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:34.122879 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p"] Jun 07 09:17:34.124698 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:17:34.124672 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e454ae3_ddd2_49e4_98cd_b409da965ef4.slice/crio-f88ae7c5f80f8405cadfd209c8cad4c7ec4b230d60167fb58a3aadf93f3d5abe WatchSource:0}: Error finding container f88ae7c5f80f8405cadfd209c8cad4c7ec4b230d60167fb58a3aadf93f3d5abe: Status 404 returned error can't find the container with id f88ae7c5f80f8405cadfd209c8cad4c7ec4b230d60167fb58a3aadf93f3d5abe Jun 07 09:17:34.649105 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:34.649068 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" event={"ID":"3e454ae3-ddd2-49e4-98cd-b409da965ef4","Type":"ContainerStarted","Data":"f88ae7c5f80f8405cadfd209c8cad4c7ec4b230d60167fb58a3aadf93f3d5abe"} Jun 07 09:17:34.855851 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:34.855763 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:36.661985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:36.661944 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" event={"ID":"3e454ae3-ddd2-49e4-98cd-b409da965ef4","Type":"ContainerStarted","Data":"c7c0bc9b915229cadd411aa7155792a2e9cfe218d4de80dc84c8ea0de60b6951"} Jun 07 09:17:36.662475 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:36.662069 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:36.683395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:36.683340 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" podStartSLOduration=1.8003222 podStartE2EDuration="3.683325077s" totalImagesPullingTime="1.883002877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:17:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:17:36.682553064 +0000 UTC m=+548.683558190" watchObservedRunningTime="2026-06-07 09:17:36.683325077 +0000 UTC m=+548.684330199" Jun 07 09:17:37.884595 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:37.884561 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:37.985577 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:37.985544 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:39.057935 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:39.057903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:39.113562 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:39.113534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:39.282717 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:39.282688 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:39.665331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:39.665299 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:39.816589 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:39.816557 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:42.361189 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:42.361158 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:44.862822 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:44.862769 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:44.963169 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:44.963143 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:47.669760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:47.669727 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-dvk8p" Jun 07 09:17:47.933794 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:47.933707 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:17:47.938709 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:47.938688 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:47.941917 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:47.941894 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4ljbr\"" Jun 07 09:17:47.948953 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:47.948923 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:17:48.008618 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.008588 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.008762 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.008692 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d57fc\" (UniqueName: \"kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.109515 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.109486 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.109707 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.109567 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d57fc\" (UniqueName: \"kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.109957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.109934 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.119477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.119451 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d57fc\" (UniqueName: \"kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n9m7j\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.250514 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.250427 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:48.392331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.392297 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:17:48.393713 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:17:48.393683 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32dcb981_fe72_45b2_b32e_374988a47404.slice/crio-e1061751f468018432c0b5e18b3e276b577e86034ed0b0c162faa523193af8e6 WatchSource:0}: Error finding container e1061751f468018432c0b5e18b3e276b577e86034ed0b0c162faa523193af8e6: Status 404 returned error can't find the container with id e1061751f468018432c0b5e18b3e276b577e86034ed0b0c162faa523193af8e6 Jun 07 09:17:48.729450 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:48.729419 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" event={"ID":"32dcb981-fe72-45b2-b32e-374988a47404","Type":"ContainerStarted","Data":"e1061751f468018432c0b5e18b3e276b577e86034ed0b0c162faa523193af8e6"} Jun 07 09:17:52.063730 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.063701 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:52.157904 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.157875 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:52.756212 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.756172 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" event={"ID":"32dcb981-fe72-45b2-b32e-374988a47404","Type":"ContainerStarted","Data":"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2"} Jun 07 09:17:52.756376 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.756254 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:17:52.794157 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.794105 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" podStartSLOduration=2.196292177 podStartE2EDuration="5.794085826s" totalImagesPullingTime="3.597793649s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:17:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:17:52.793450544 +0000 UTC m=+564.794455667" watchObservedRunningTime="2026-06-07 09:17:52.794085826 +0000 UTC m=+564.795090950" Jun 07 09:17:52.842765 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.842727 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:52.943539 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:52.943507 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:56.272151 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:56.272116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:56.372349 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:56.372306 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:17:57.400673 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.400610 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-768fd7998b-vlcnv" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerName="console" containerID="cri-o://86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96" gracePeriod=15 Jun 07 09:17:57.431499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.431470 2562 patch_prober.go:28] interesting pod/console-768fd7998b-vlcnv container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused" start-of-body= Jun 07 09:17:57.431607 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.431549 2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-768fd7998b-vlcnv" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerName="console" probeResult="failure" output="Get \"https://10.134.0.19:8443/health\": dial tcp 10.134.0.19:8443: connect: connection refused" Jun 07 09:17:57.644079 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.644055 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:17:57.689960 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.689928 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690093 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.689986 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690161 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690106 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690218 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690162 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jw679\" (UniqueName: \"kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690326 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690227 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690326 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690255 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690326 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config\") pod \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\" (UID: \"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9\") " Jun 07 09:17:57.690601 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690559 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca" (OuterVolumeSpecName: "service-ca") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:17:57.691026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690691 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:17:57.691026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690842 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:17:57.691026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.690910 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config" (OuterVolumeSpecName: "console-config") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:17:57.692140 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.692115 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:17:57.692407 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.692381 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679" (OuterVolumeSpecName: "kube-api-access-jw679") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "kube-api-access-jw679". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:17:57.692507 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.692474 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" (UID: "eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 07 09:17:57.788108 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.788074 2562 generic.go:356] "Generic (PLEG): container finished" podID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerID="86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96" exitCode=0 Jun 07 09:17:57.788214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.788121 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768fd7998b-vlcnv" Jun 07 09:17:57.788214 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.788161 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fd7998b-vlcnv" event={"ID":"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9","Type":"ContainerDied","Data":"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96"} Jun 07 09:17:57.788288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.788231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768fd7998b-vlcnv" event={"ID":"eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9","Type":"ContainerDied","Data":"19ede2fad0624cb608d16e0d245f2c47da20adc99481a0c43b3cef6e9ecc8861"} Jun 07 09:17:57.788288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.788252 2562 scope.go:117] "RemoveContainer" containerID="86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96" Jun 07 09:17:57.790988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.790970 2562 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-serving-cert\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.790991 2562 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-oauth-config\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.791002 2562 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-service-ca\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.791011 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jw679\" (UniqueName: \"kubernetes.io/projected/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-kube-api-access-jw679\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.791020 2562 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-trusted-ca-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.791029 2562 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-oauth-serving-cert\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.791071 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.791038 2562 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9-console-config\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:17:57.797637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.797620 2562 scope.go:117] "RemoveContainer" containerID="86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96" Jun 07 09:17:57.797930 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:17:57.797913 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96\": container with ID starting with 86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96 not found: ID does not exist" containerID="86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96" Jun 07 09:17:57.797990 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.797939 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96"} err="failed to get container status \"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96\": rpc error: code = NotFound desc = could not find container \"86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96\": container with ID starting with 86a4f4a6c380072f664d853dda812946966836a07ed86c54639166e8dab2af96 not found: ID does not exist" Jun 07 09:17:57.810904 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.810884 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:17:57.814097 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:57.814073 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768fd7998b-vlcnv"] Jun 07 09:17:58.412557 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:17:58.412522 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" path="/var/lib/kubelet/pods/eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9/volumes" Jun 07 09:18:00.061082 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:00.061048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:00.161579 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:00.161534 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:00.265616 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:00.265581 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:00.319488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:00.319419 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:02.782742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:02.782705 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:02.882859 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:02.882794 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:03.752164 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:03.752130 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:18:03.752409 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:03.752341 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" podUID="32dcb981-fe72-45b2-b32e-374988a47404" containerName="manager" containerID="cri-o://af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2" gracePeriod=10 Jun 07 09:18:03.754334 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:03.754308 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:18:04.007753 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.007693 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:18:04.040366 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.040339 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume\") pod \"32dcb981-fe72-45b2-b32e-374988a47404\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " Jun 07 09:18:04.040473 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.040409 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d57fc\" (UniqueName: \"kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc\") pod \"32dcb981-fe72-45b2-b32e-374988a47404\" (UID: \"32dcb981-fe72-45b2-b32e-374988a47404\") " Jun 07 09:18:04.040775 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.040754 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "32dcb981-fe72-45b2-b32e-374988a47404" (UID: "32dcb981-fe72-45b2-b32e-374988a47404"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:18:04.042271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.042248 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc" (OuterVolumeSpecName: "kube-api-access-d57fc") pod "32dcb981-fe72-45b2-b32e-374988a47404" (UID: "32dcb981-fe72-45b2-b32e-374988a47404"). InnerVolumeSpecName "kube-api-access-d57fc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:18:04.141943 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.141915 2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32dcb981-fe72-45b2-b32e-374988a47404-extensions-socket-volume\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:18:04.141943 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.141939 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d57fc\" (UniqueName: \"kubernetes.io/projected/32dcb981-fe72-45b2-b32e-374988a47404-kube-api-access-d57fc\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:18:04.827742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.827652 2562 generic.go:356] "Generic (PLEG): container finished" podID="32dcb981-fe72-45b2-b32e-374988a47404" containerID="af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2" exitCode=0 Jun 07 09:18:04.827742 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.827696 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" Jun 07 09:18:04.827965 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.827737 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" event={"ID":"32dcb981-fe72-45b2-b32e-374988a47404","Type":"ContainerDied","Data":"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2"} Jun 07 09:18:04.827965 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.827786 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j" event={"ID":"32dcb981-fe72-45b2-b32e-374988a47404","Type":"ContainerDied","Data":"e1061751f468018432c0b5e18b3e276b577e86034ed0b0c162faa523193af8e6"} Jun 07 09:18:04.827965 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.827824 2562 scope.go:117] "RemoveContainer" containerID="af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2" Jun 07 09:18:04.836832 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.836792 2562 scope.go:117] "RemoveContainer" containerID="af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2" Jun 07 09:18:04.837100 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:18:04.837080 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2\": container with ID starting with af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2 not found: ID does not exist" containerID="af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2" Jun 07 09:18:04.837146 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.837110 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2"} err="failed to get container status \"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2\": rpc error: code = NotFound desc = could not find container \"af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2\": container with ID starting with af12836476488dba2bd65e0e22ae8dd13fb17611666cf4d6611c506c01873fb2 not found: ID does not exist" Jun 07 09:18:04.850576 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.850543 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:18:04.853943 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:04.853919 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n9m7j"] Jun 07 09:18:06.411411 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:06.411378 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32dcb981-fe72-45b2-b32e-374988a47404" path="/var/lib/kubelet/pods/32dcb981-fe72-45b2-b32e-374988a47404/volumes" Jun 07 09:18:09.265485 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:09.265455 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:09.362395 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:09.362364 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:14.012414 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:14.012376 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:14.116712 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:14.116675 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:17.809679 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:17.809644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:17.910546 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:17.910508 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:18.210552 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:18.210512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:18.266998 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:18.266953 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:20.020361 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.020323 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2"] Jun 07 09:18:20.020988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.020923 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerName="console" Jun 07 09:18:20.020988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.020944 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerName="console" Jun 07 09:18:20.020988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.020965 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32dcb981-fe72-45b2-b32e-374988a47404" containerName="manager" Jun 07 09:18:20.020988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.020974 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="32dcb981-fe72-45b2-b32e-374988a47404" containerName="manager" Jun 07 09:18:20.021235 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.021063 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="32dcb981-fe72-45b2-b32e-374988a47404" containerName="manager" Jun 07 09:18:20.021235 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.021080 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb2bf123-bf3c-4b4c-bfbd-ad8a55d792f9" containerName="console" Jun 07 09:18:20.025557 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.025535 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.028574 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.028548 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-nrkl5\"" Jun 07 09:18:20.039173 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.039151 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2"] Jun 07 09:18:20.055514 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.055485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:20.112478 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.112450 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:20.168963 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.168924 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.168994 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169027 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169048 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169068 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169137 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169107 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169154 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169171 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.169413 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.169191 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p88fz\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-kube-api-access-p88fz\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270127 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270095 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270127 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270130 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270149 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270169 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270293 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270371 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270401 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270681 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270548 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-certs\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270681 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270440 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p88fz\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-kube-api-access-p88fz\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270681 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270672 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270827 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270746 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-workload-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.270984 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270959 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-data\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.271110 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.270957 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.271110 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.271074 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-credential-socket\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.272928 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.272908 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-envoy\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.273258 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.273226 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-podinfo\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.279925 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.279905 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-istio-token\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.280359 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.280326 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p88fz\" (UniqueName: \"kubernetes.io/projected/bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49-kube-api-access-p88fz\") pod \"maas-default-gateway-openshift-default-687ff6996-6lzr2\" (UID: \"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49\") " pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.339064 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.339038 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:20.482511 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.482481 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2"] Jun 07 09:18:20.483744 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:18:20.483718 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7864dc_3cd8_4dd5_a5cd_2d682fd18a49.slice/crio-154c1ecdf5494b85955e712e8b6ac2fa64645434415bf3a425d2c3a7651335e3 WatchSource:0}: Error finding container 154c1ecdf5494b85955e712e8b6ac2fa64645434415bf3a425d2c3a7651335e3: Status 404 returned error can't find the container with id 154c1ecdf5494b85955e712e8b6ac2fa64645434415bf3a425d2c3a7651335e3 Jun 07 09:18:20.913329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:20.913290 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" event={"ID":"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49","Type":"ContainerStarted","Data":"154c1ecdf5494b85955e712e8b6ac2fa64645434415bf3a425d2c3a7651335e3"} Jun 07 09:18:22.216849 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:22.216797 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:22.274536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:22.274506 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:23.002887 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.002850 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:18:23.002967 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.002923 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:18:23.002967 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.002953 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:18:23.436783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.436756 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:23.931280 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.931249 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" event={"ID":"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49","Type":"ContainerStarted","Data":"c2b067da01c81188c911ef3d27b4a6c2c1ac3708f205d3c99fd110abca9e3626"} Jun 07 09:18:23.957737 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:23.957692 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" podStartSLOduration=2.441301584 podStartE2EDuration="4.957678681s" totalImagesPullingTime="2.516377097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:18:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:18:23.954753802 +0000 UTC m=+595.955758942" watchObservedRunningTime="2026-06-07 09:18:23.957678681 +0000 UTC m=+595.958683876" Jun 07 09:18:24.339513 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.339433 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:24.344482 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.344453 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:24.354104 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.354075 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:24.357691 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.357672 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.358609 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.358585 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:24.362207 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.362184 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 07 09:18:24.362306 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.362287 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-jdchw\"" Jun 07 09:18:24.367104 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.367081 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:24.456975 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.456945 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:24.459656 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.459626 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:24.506067 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.506037 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsqjt\" (UniqueName: \"kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.506190 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.506100 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.514026 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.513998 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:24.606920 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.606842 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.607079 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.606934 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsqjt\" (UniqueName: \"kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.607504 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.607480 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.614652 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.614631 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsqjt\" (UniqueName: \"kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt\") pod \"limitador-limitador-64c94fd64f-865l8\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.668765 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.668744 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:24.800367 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.800342 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:24.801356 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:18:24.801326 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd638d92d_b3a2_4dc4_b19a_192713d5ac78.slice/crio-c3afed6cfdb2f95260c8342c04dca96a399e6d68bf03d8bcc06ebb881e92f13e WatchSource:0}: Error finding container c3afed6cfdb2f95260c8342c04dca96a399e6d68bf03d8bcc06ebb881e92f13e: Status 404 returned error can't find the container with id c3afed6cfdb2f95260c8342c04dca96a399e6d68bf03d8bcc06ebb881e92f13e Jun 07 09:18:24.937900 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.937865 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" event={"ID":"d638d92d-b3a2-4dc4-b19a-192713d5ac78","Type":"ContainerStarted","Data":"c3afed6cfdb2f95260c8342c04dca96a399e6d68bf03d8bcc06ebb881e92f13e"} Jun 07 09:18:24.938104 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.937974 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:24.939167 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:24.939145 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:18:25.221008 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:25.220925 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:25.321488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:25.321459 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:27.957582 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:27.957546 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" event={"ID":"d638d92d-b3a2-4dc4-b19a-192713d5ac78","Type":"ContainerStarted","Data":"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8"} Jun 07 09:18:27.958035 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:27.957608 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:27.976534 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:27.976485 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" podStartSLOduration=1.320369668 podStartE2EDuration="3.976470369s" totalImagesPullingTime="2.656100701s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:18:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:18:27.972771298 +0000 UTC m=+599.973776424" watchObservedRunningTime="2026-06-07 09:18:27.976470369 +0000 UTC m=+599.977475492" Jun 07 09:18:28.387831 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:28.387739 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:18:28.390232 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:28.390216 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:18:28.392344 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:28.392326 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:18:28.400634 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:28.400615 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:18:30.104666 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:30.104637 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:30.162047 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:30.162019 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:33.627330 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:33.627295 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:33.727376 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:33.727339 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:38.963970 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:38.963893 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:39.632084 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:39.632050 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:39.632285 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:39.632248 2562 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" podUID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" containerName="limitador" containerID="cri-o://290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8" gracePeriod=30 Jun 07 09:18:40.182956 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.182933 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:40.337640 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.337551 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xsqjt\" (UniqueName: \"kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt\") pod \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " Jun 07 09:18:40.337640 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.337592 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file\") pod \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\" (UID: \"d638d92d-b3a2-4dc4-b19a-192713d5ac78\") " Jun 07 09:18:40.338020 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.337994 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file" (OuterVolumeSpecName: "config-file") pod "d638d92d-b3a2-4dc4-b19a-192713d5ac78" (UID: "d638d92d-b3a2-4dc4-b19a-192713d5ac78"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 07 09:18:40.339695 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.339668 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt" (OuterVolumeSpecName: "kube-api-access-xsqjt") pod "d638d92d-b3a2-4dc4-b19a-192713d5ac78" (UID: "d638d92d-b3a2-4dc4-b19a-192713d5ac78"). InnerVolumeSpecName "kube-api-access-xsqjt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:18:40.439001 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.438963 2562 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/d638d92d-b3a2-4dc4-b19a-192713d5ac78-config-file\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:18:40.439001 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:40.438987 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xsqjt\" (UniqueName: \"kubernetes.io/projected/d638d92d-b3a2-4dc4-b19a-192713d5ac78-kube-api-access-xsqjt\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:18:41.024286 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.024253 2562 generic.go:356] "Generic (PLEG): container finished" podID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" containerID="290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8" exitCode=0 Jun 07 09:18:41.024479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.024296 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" Jun 07 09:18:41.024479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.024316 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" event={"ID":"d638d92d-b3a2-4dc4-b19a-192713d5ac78","Type":"ContainerDied","Data":"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8"} Jun 07 09:18:41.024479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.024357 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-865l8" event={"ID":"d638d92d-b3a2-4dc4-b19a-192713d5ac78","Type":"ContainerDied","Data":"c3afed6cfdb2f95260c8342c04dca96a399e6d68bf03d8bcc06ebb881e92f13e"} Jun 07 09:18:41.024479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.024378 2562 scope.go:117] "RemoveContainer" containerID="290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8" Jun 07 09:18:41.033042 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.033022 2562 scope.go:117] "RemoveContainer" containerID="290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8" Jun 07 09:18:41.033318 ip-10-0-128-125 kubenswrapper[2562]: E0607 09:18:41.033292 2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8\": container with ID starting with 290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8 not found: ID does not exist" containerID="290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8" Jun 07 09:18:41.033383 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.033319 2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8"} err="failed to get container status \"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8\": rpc error: code = NotFound desc = could not find container \"290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8\": container with ID starting with 290a82ce48c5a88975ab583bcc1ded92f807b48dcd16f2e8fabce475b9c6c5c8 not found: ID does not exist" Jun 07 09:18:41.042511 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.042484 2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:41.045629 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:41.045605 2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-865l8"] Jun 07 09:18:42.412534 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:42.412503 2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" path="/var/lib/kubelet/pods/d638d92d-b3a2-4dc4-b19a-192713d5ac78/volumes" Jun 07 09:18:45.553751 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.553713 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-snbws"] Jun 07 09:18:45.554345 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.554272 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" containerName="limitador" Jun 07 09:18:45.554345 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.554294 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" containerName="limitador" Jun 07 09:18:45.554476 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.554408 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d638d92d-b3a2-4dc4-b19a-192713d5ac78" containerName="limitador" Jun 07 09:18:45.559251 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.559223 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.562253 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.562225 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-snbws"] Jun 07 09:18:45.562386 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.562257 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jun 07 09:18:45.579916 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.579858 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:45.679321 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.679291 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h4p2\" (UniqueName: \"kubernetes.io/projected/0a4a3251-c19a-468d-bffa-22112e7e5b72-kube-api-access-7h4p2\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.679467 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.679368 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0a4a3251-c19a-468d-bffa-22112e7e5b72-data\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.779913 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.779885 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0a4a3251-c19a-468d-bffa-22112e7e5b72-data\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.780031 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.779935 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h4p2\" (UniqueName: \"kubernetes.io/projected/0a4a3251-c19a-468d-bffa-22112e7e5b72-kube-api-access-7h4p2\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.780246 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.780228 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/0a4a3251-c19a-468d-bffa-22112e7e5b72-data\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.788414 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.788388 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h4p2\" (UniqueName: \"kubernetes.io/projected/0a4a3251-c19a-468d-bffa-22112e7e5b72-kube-api-access-7h4p2\") pod \"postgres-69fb4594cf-snbws\" (UID: \"0a4a3251-c19a-468d-bffa-22112e7e5b72\") " pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.873010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.872958 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:45.999475 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:45.999441 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-snbws"] Jun 07 09:18:46.000636 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:18:46.000603 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a4a3251_c19a_468d_bffa_22112e7e5b72.slice/crio-291abd9f8129087a26b4ea1f1f5b11be62953bf1064843a0c9b6b1df6cb5f7e8 WatchSource:0}: Error finding container 291abd9f8129087a26b4ea1f1f5b11be62953bf1064843a0c9b6b1df6cb5f7e8: Status 404 returned error can't find the container with id 291abd9f8129087a26b4ea1f1f5b11be62953bf1064843a0c9b6b1df6cb5f7e8 Jun 07 09:18:46.053375 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:46.053340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-snbws" event={"ID":"0a4a3251-c19a-468d-bffa-22112e7e5b72","Type":"ContainerStarted","Data":"291abd9f8129087a26b4ea1f1f5b11be62953bf1064843a0c9b6b1df6cb5f7e8"} Jun 07 09:18:47.155928 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:47.155844 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:47.207340 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:47.207286 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:49.073664 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:49.073629 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-snbws" event={"ID":"0a4a3251-c19a-468d-bffa-22112e7e5b72","Type":"ContainerStarted","Data":"b8b709d18b655100de2b5c513edcf43f600cc349ec2546aa1ff7d28fda72f4ba"} Jun 07 09:18:49.074155 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:49.073723 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:49.090801 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:49.090750 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-snbws" podStartSLOduration=1.858278361 podStartE2EDuration="4.090734823s" totalImagesPullingTime="2.232456462s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:18:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:18:49.089832227 +0000 UTC m=+621.090837352" watchObservedRunningTime="2026-06-07 09:18:49.090734823 +0000 UTC m=+621.091739946" Jun 07 09:18:53.194587 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:53.194551 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:53.296324 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:53.296284 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:55.112403 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:55.112377 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-snbws" Jun 07 09:18:55.417388 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:55.417346 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:55.514333 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:55.514305 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:55.732901 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:55.732794 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:18:56.514294 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.514262 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g"] Jun 07 09:18:56.522151 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.522129 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.528845 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.525328 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-7zdpv\"" Jun 07 09:18:56.528845 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.527073 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 07 09:18:56.528845 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.527597 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 07 09:18:56.530009 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.529103 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g"] Jun 07 09:18:56.573660 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.573626 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.573785 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.573668 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.573850 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.573782 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtc8k\" (UniqueName: \"kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.675274 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.675240 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.675430 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.675288 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.675430 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.675349 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtc8k\" (UniqueName: \"kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.675672 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.675653 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.675722 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.675694 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.684379 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.684354 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtc8k\" (UniqueName: \"kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k\") pod \"0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.840760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.840673 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:18:56.972886 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:56.972853 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g"] Jun 07 09:18:56.973869 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:18:56.973843 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea09e5c1_aca1_4864_8ff5_681d1e69e1e3.slice/crio-b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87 WatchSource:0}: Error finding container b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87: Status 404 returned error can't find the container with id b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87 Jun 07 09:18:57.115957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:57.115879 2562 generic.go:356] "Generic (PLEG): container finished" podID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerID="7f95e785f7ebd82c3df8598507a4c90ce460d9e0134839ecb4bf1b359bf94f3e" exitCode=0 Jun 07 09:18:57.115957 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:57.115926 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" event={"ID":"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3","Type":"ContainerDied","Data":"7f95e785f7ebd82c3df8598507a4c90ce460d9e0134839ecb4bf1b359bf94f3e"} Jun 07 09:18:57.116262 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:57.116237 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" event={"ID":"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3","Type":"ContainerStarted","Data":"b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87"} Jun 07 09:18:58.124115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:58.124034 2562 generic.go:356] "Generic (PLEG): container finished" podID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerID="05e8a54e1659b32e9cc06cfb6f3933c7a07286862fd9dfd8e91cbb9271456365" exitCode=0 Jun 07 09:18:58.124479 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:58.124122 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" event={"ID":"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3","Type":"ContainerDied","Data":"05e8a54e1659b32e9cc06cfb6f3933c7a07286862fd9dfd8e91cbb9271456365"} Jun 07 09:18:59.131585 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:59.131548 2562 generic.go:356] "Generic (PLEG): container finished" podID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerID="5a0f5955cb3fdf4c7166fbf4cd856667ab8a202bc0e9ef087c84f9d517074ec4" exitCode=0 Jun 07 09:18:59.131967 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:18:59.131640 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" event={"ID":"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3","Type":"ContainerDied","Data":"5a0f5955cb3fdf4c7166fbf4cd856667ab8a202bc0e9ef087c84f9d517074ec4"} Jun 07 09:19:00.282317 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.282295 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:19:00.405372 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.405293 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle\") pod \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " Jun 07 09:19:00.405372 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.405358 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtc8k\" (UniqueName: \"kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k\") pod \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " Jun 07 09:19:00.405593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.405442 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util\") pod \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\" (UID: \"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3\") " Jun 07 09:19:00.405932 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.405898 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle" (OuterVolumeSpecName: "bundle") pod "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" (UID: "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:19:00.407472 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.407450 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k" (OuterVolumeSpecName: "kube-api-access-gtc8k") pod "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" (UID: "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3"). InnerVolumeSpecName "kube-api-access-gtc8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:19:00.412004 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.411979 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util" (OuterVolumeSpecName: "util") pod "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" (UID: "ea09e5c1-aca1-4864-8ff5-681d1e69e1e3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 07 09:19:00.506173 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.506146 2562 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-util\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:19:00.506173 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.506169 2562 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-bundle\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:19:00.506322 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:00.506178 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtc8k\" (UniqueName: \"kubernetes.io/projected/ea09e5c1-aca1-4864-8ff5-681d1e69e1e3-kube-api-access-gtc8k\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:19:01.150468 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:01.150430 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" event={"ID":"ea09e5c1-aca1-4864-8ff5-681d1e69e1e3","Type":"ContainerDied","Data":"b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87"} Jun 07 09:19:01.150637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:01.150508 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b57231b1f4e5c7f356452efcc2aca17c3842245aec621ab88d79bb1e5d5d1a87" Jun 07 09:19:01.150637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:01.150444 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/0228a3cc79630676fae641e6a38dab1752e85a08a4a9edc59ccf0da748r2h5g" Jun 07 09:19:03.818369 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818328 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-snpm7"] Jun 07 09:19:03.818899 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818878 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="pull" Jun 07 09:19:03.818986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818902 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="pull" Jun 07 09:19:03.818986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818927 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="util" Jun 07 09:19:03.818986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818935 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="util" Jun 07 09:19:03.818986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818948 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="extract" Jun 07 09:19:03.818986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.818956 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="extract" Jun 07 09:19:03.819237 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.819084 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea09e5c1-aca1-4864-8ff5-681d1e69e1e3" containerName="extract" Jun 07 09:19:03.822287 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.822263 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" Jun 07 09:19:03.825548 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.825526 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 07 09:19:03.827229 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.827208 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"keycloak-operator-dockercfg-czjvf\"" Jun 07 09:19:03.827468 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.827313 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 07 09:19:03.831251 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.831230 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-snpm7"] Jun 07 09:19:03.935528 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:03.935502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nt9tr\" (UniqueName: \"kubernetes.io/projected/e65470fe-0fcb-4926-ba24-8b1fe1002812-kube-api-access-nt9tr\") pod \"keycloak-operator-d67dbbb48-snpm7\" (UID: \"e65470fe-0fcb-4926-ba24-8b1fe1002812\") " pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" Jun 07 09:19:04.036895 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:04.036866 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nt9tr\" (UniqueName: \"kubernetes.io/projected/e65470fe-0fcb-4926-ba24-8b1fe1002812-kube-api-access-nt9tr\") pod \"keycloak-operator-d67dbbb48-snpm7\" (UID: \"e65470fe-0fcb-4926-ba24-8b1fe1002812\") " pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" Jun 07 09:19:04.045476 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:04.045449 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nt9tr\" (UniqueName: \"kubernetes.io/projected/e65470fe-0fcb-4926-ba24-8b1fe1002812-kube-api-access-nt9tr\") pod \"keycloak-operator-d67dbbb48-snpm7\" (UID: \"e65470fe-0fcb-4926-ba24-8b1fe1002812\") " pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" Jun 07 09:19:04.133972 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:04.133919 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" Jun 07 09:19:04.274420 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:04.274397 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["keycloak-system/keycloak-operator-d67dbbb48-snpm7"] Jun 07 09:19:04.276850 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:19:04.276825 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65470fe_0fcb_4926_ba24_8b1fe1002812.slice/crio-814774c1c5584bf7146fac6664ec9cebe37dc83e274bc35bed469c526d427f1f WatchSource:0}: Error finding container 814774c1c5584bf7146fac6664ec9cebe37dc83e274bc35bed469c526d427f1f: Status 404 returned error can't find the container with id 814774c1c5584bf7146fac6664ec9cebe37dc83e274bc35bed469c526d427f1f Jun 07 09:19:05.176620 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:05.176580 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" event={"ID":"e65470fe-0fcb-4926-ba24-8b1fe1002812","Type":"ContainerStarted","Data":"814774c1c5584bf7146fac6664ec9cebe37dc83e274bc35bed469c526d427f1f"} Jun 07 09:19:10.209340 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:10.209305 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" event={"ID":"e65470fe-0fcb-4926-ba24-8b1fe1002812","Type":"ContainerStarted","Data":"109c60e5bde72d76e21e3f858b5885c0ecde0c892df5017084faef9e22c7e9d5"} Jun 07 09:19:10.228076 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:10.228028 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="keycloak-system/keycloak-operator-d67dbbb48-snpm7" podStartSLOduration=1.767918705 podStartE2EDuration="7.228012998s" totalImagesPullingTime="5.460094293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:19:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:19:10.225243871 +0000 UTC m=+642.226249000" watchObservedRunningTime="2026-06-07 09:19:10.228012998 +0000 UTC m=+642.229018192" Jun 07 09:19:18.487774 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:18.487734 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:18.544749 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:18.544713 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:48.991666 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:48.991634 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:49.134197 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:49.134163 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:54.844800 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:54.844764 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:54.946164 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:54.946134 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:55.007202 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:55.007169 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:55.062978 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:55.062939 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:58.333691 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:58.333660 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:19:58.464053 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:19:58.464022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:20:27.718784 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:27.718751 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:20:27.853520 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:27.853481 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:20:28.247831 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:28.247789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:20:28.344731 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:28.344696 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:20:36.698188 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.698158 2562 generic.go:356] "Generic (PLEG): container finished" podID="bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49" containerID="c2b067da01c81188c911ef3d27b4a6c2c1ac3708f205d3c99fd110abca9e3626" exitCode=0 Jun 07 09:20:36.698598 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.698231 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" event={"ID":"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49","Type":"ContainerDied","Data":"c2b067da01c81188c911ef3d27b4a6c2c1ac3708f205d3c99fd110abca9e3626"} Jun 07 09:20:36.698649 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.698610 2562 scope.go:117] "RemoveContainer" containerID="c2b067da01c81188c911ef3d27b4a6c2c1ac3708f205d3c99fd110abca9e3626" Jun 07 09:20:36.699620 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.699592 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:20:36.699672 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.699660 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:20:36.699711 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:36.699693 2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jun 07 09:20:37.706024 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:37.705989 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" event={"ID":"bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49","Type":"ContainerStarted","Data":"dbf4ae5bfdb32a71d8985da2c7b14e0661bfc847135bb05aff7113b3674d4ae6"} Jun 07 09:20:38.341363 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:38.341327 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:20:38.341851 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:38.341824 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-6lzr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jun 07 09:20:38.342045 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:38.342019 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" podUID="bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jun 07 09:20:39.340122 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:39.340086 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-6lzr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jun 07 09:20:39.340498 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:39.340147 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" podUID="bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jun 07 09:20:39.938414 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:39.938378 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:20:40.339536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:40.339443 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-6lzr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jun 07 09:20:40.339536 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:40.339519 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" podUID="bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jun 07 09:20:41.339415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:41.339372 2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-687ff6996-6lzr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" start-of-body= Jun 07 09:20:41.339790 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:41.339431 2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" podUID="bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.43:15021/healthz/ready\": dial tcp 10.134.0.43:15021: connect: connection refused" Jun 07 09:20:42.344339 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:42.344309 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:20:42.345356 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:20:42.345333 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-687ff6996-6lzr2" Jun 07 09:21:17.255881 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:17.255846 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:17.359369 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:17.359338 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:24.210523 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:24.210485 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:42.908889 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:42.908789 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:48.374010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.373977 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8"] Jun 07 09:21:48.377763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.377742 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.383070 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.382219 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 07 09:21:48.383070 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.382218 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jun 07 09:21:48.383070 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.382222 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-htb4v\"" Jun 07 09:21:48.383070 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.382952 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 07 09:21:48.386888 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.386842 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8"] Jun 07 09:21:48.404253 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.404224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:48.496364 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496341 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.496499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496386 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.496499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496459 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.496499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496486 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06908d4f-8e00-4244-8cb4-527e7fbfd98b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.496604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496505 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.496604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.496531 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t467v\" (UniqueName: \"kubernetes.io/projected/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kube-api-access-t467v\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.597762 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.597733 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.597985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.597769 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06908d4f-8e00-4244-8cb4-527e7fbfd98b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.597985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.597818 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.597985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.597865 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t467v\" (UniqueName: \"kubernetes.io/projected/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kube-api-access-t467v\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.597985 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.597952 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.598223 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.598013 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.598372 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.598351 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-home\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.598477 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.598454 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.598526 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.598507 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kserve-provision-location\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.600160 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.600141 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/06908d4f-8e00-4244-8cb4-527e7fbfd98b-dshm\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.600304 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.600286 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/06908d4f-8e00-4244-8cb4-527e7fbfd98b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.605657 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.605638 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t467v\" (UniqueName: \"kubernetes.io/projected/06908d4f-8e00-4244-8cb4-527e7fbfd98b-kube-api-access-t467v\") pod \"e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8\" (UID: \"06908d4f-8e00-4244-8cb4-527e7fbfd98b\") " pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.690198 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.690172 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:48.822984 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.822949 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8"] Jun 07 09:21:48.824427 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:21:48.824394 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06908d4f_8e00_4244_8cb4_527e7fbfd98b.slice/crio-23310d561bb415f1bbd618753592972e84a3ebbaca81bc84906a81f2ef5f4171 WatchSource:0}: Error finding container 23310d561bb415f1bbd618753592972e84a3ebbaca81bc84906a81f2ef5f4171: Status 404 returned error can't find the container with id 23310d561bb415f1bbd618753592972e84a3ebbaca81bc84906a81f2ef5f4171 Jun 07 09:21:48.826728 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:48.826707 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:21:49.104940 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:49.104846 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" event={"ID":"06908d4f-8e00-4244-8cb4-527e7fbfd98b","Type":"ContainerStarted","Data":"23310d561bb415f1bbd618753592972e84a3ebbaca81bc84906a81f2ef5f4171"} Jun 07 09:21:52.127444 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:52.127409 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" event={"ID":"06908d4f-8e00-4244-8cb4-527e7fbfd98b","Type":"ContainerStarted","Data":"d7c14f3377860686912016f32d84621aa604c228e05b09e8eac84d4ba0cfe10b"} Jun 07 09:21:53.049091 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:53.049059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:53.134727 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:53.134697 2562 generic.go:356] "Generic (PLEG): container finished" podID="06908d4f-8e00-4244-8cb4-527e7fbfd98b" containerID="d7c14f3377860686912016f32d84621aa604c228e05b09e8eac84d4ba0cfe10b" exitCode=0 Jun 07 09:21:53.135127 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:53.134784 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" event={"ID":"06908d4f-8e00-4244-8cb4-527e7fbfd98b","Type":"ContainerDied","Data":"d7c14f3377860686912016f32d84621aa604c228e05b09e8eac84d4ba0cfe10b"} Jun 07 09:21:55.150715 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:55.150674 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" event={"ID":"06908d4f-8e00-4244-8cb4-527e7fbfd98b","Type":"ContainerStarted","Data":"bbb087e60815cb3d358a090d70494c0dfe3eb63d9f181cb635f439c5b3e016fa"} Jun 07 09:21:55.173278 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:55.173214 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" podStartSLOduration=2.0414362 podStartE2EDuration="7.173192592s" totalImagesPullingTime="3.875749699s" totalInitContainerRuntime="1.256006693s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:21:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:21:55.172151561 +0000 UTC m=+807.173156685" watchObservedRunningTime="2026-06-07 09:21:55.173192592 +0000 UTC m=+807.174197716" Jun 07 09:21:55.205195 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:55.205147 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:21:58.690310 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:58.690278 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:58.690728 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:58.690337 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:58.702727 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:58.702699 2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:21:59.183840 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:21:59.183789 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8" Jun 07 09:22:02.694206 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:22:02.694172 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:22:04.968905 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:22:04.968871 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:22:10.313409 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:22:10.313354 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:23:19.829390 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:19.829350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:23:28.437326 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:28.437301 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:23:28.437763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:28.437704 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:23:28.441875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:28.441854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:23:28.442081 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:28.442066 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:23:36.886790 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:36.886749 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:23:36.986893 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:36.986826 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:23:38.080109 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:38.080079 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:23:38.178275 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:23:38.178224 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:24:36.854658 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:24:36.854612 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:05.968977 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:05.968857 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:07.715047 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:07.715006 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:34.969864 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:34.969801 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:49.680872 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:49.680833 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:49.782340 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:49.782298 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:25:58.186419 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:25:58.186383 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:26:04.197115 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:26:04.197076 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:26:04.256079 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:26:04.256048 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:26:36.380950 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:26:36.380903 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:26:36.477695 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:26:36.477662 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:00.337093 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:00.337022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:05.964069 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:05.964023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:06.084010 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:06.083972 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:06.173981 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:06.173931 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:06.325664 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:06.325530 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:06.424822 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:06.424775 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:22.848722 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:22.848691 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:22.958005 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:22.957969 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:23.064207 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:23.064174 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:28.675900 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:28.675851 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:28.776166 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:28.776116 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:36.826950 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:36.826914 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:36.925521 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:36.925486 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:37.024874 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:37.024842 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:37.126538 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:37.126463 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:53.538660 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:53.538580 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:53.657345 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:53.657307 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:53.779792 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:53.779752 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:59.358515 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:59.358484 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:27:59.413665 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:27:59.413638 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:19.792050 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:19.792018 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:19.892121 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:19.892083 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:20.001488 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:20.001457 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:25.663288 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:25.663255 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:25.762532 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:25.762499 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:25.864407 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:25.864378 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:26.002251 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:26.002213 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:28.483661 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:28.483625 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:28:28.486061 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:28.486028 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:28:28.488621 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:28.488601 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:28:28.491225 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:28.491201 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:28:31.613124 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:31.613084 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:28:31.713490 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:28:31.713461 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:30:00.142450 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.142414 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29680410-mm4ts"] Jun 07 09:30:00.144783 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.144766 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:00.147875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.147852 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9svnk\"" Jun 07 09:30:00.159307 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.159284 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29680410-mm4ts"] Jun 07 09:30:00.221790 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.221752 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ppnkq\" (UniqueName: \"kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq\") pod \"maas-api-key-cleanup-29680410-mm4ts\" (UID: \"2bccc230-0cb8-4fc5-9746-d6060d49f9eb\") " pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:00.322439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.322407 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ppnkq\" (UniqueName: \"kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq\") pod \"maas-api-key-cleanup-29680410-mm4ts\" (UID: \"2bccc230-0cb8-4fc5-9746-d6060d49f9eb\") " pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:00.331594 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.331559 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ppnkq\" (UniqueName: \"kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq\") pod \"maas-api-key-cleanup-29680410-mm4ts\" (UID: \"2bccc230-0cb8-4fc5-9746-d6060d49f9eb\") " pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:00.454560 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.454532 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:00.589136 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.589109 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29680410-mm4ts"] Jun 07 09:30:00.591132 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:30:00.591099 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bccc230_0cb8_4fc5_9746_d6060d49f9eb.slice/crio-1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab WatchSource:0}: Error finding container 1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab: Status 404 returned error can't find the container with id 1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab Jun 07 09:30:00.593464 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.593443 2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 07 09:30:00.890423 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:00.890340 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" event={"ID":"2bccc230-0cb8-4fc5-9746-d6060d49f9eb","Type":"ContainerStarted","Data":"1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab"} Jun 07 09:30:48.051729 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:48.051692 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:30:56.200397 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:56.200362 2562 generic.go:356] "Generic (PLEG): container finished" podID="2bccc230-0cb8-4fc5-9746-d6060d49f9eb" containerID="4a602d74250397a997b8187202ef3788ed1d0606c9b7a357af45332b037ae53b" exitCode=0 Jun 07 09:30:56.200769 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:56.200444 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" event={"ID":"2bccc230-0cb8-4fc5-9746-d6060d49f9eb","Type":"ContainerDied","Data":"4a602d74250397a997b8187202ef3788ed1d0606c9b7a357af45332b037ae53b"} Jun 07 09:30:57.341406 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:57.341380 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:57.391889 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:57.391861 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ppnkq\" (UniqueName: \"kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq\") pod \"2bccc230-0cb8-4fc5-9746-d6060d49f9eb\" (UID: \"2bccc230-0cb8-4fc5-9746-d6060d49f9eb\") " Jun 07 09:30:57.393979 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:57.393954 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq" (OuterVolumeSpecName: "kube-api-access-ppnkq") pod "2bccc230-0cb8-4fc5-9746-d6060d49f9eb" (UID: "2bccc230-0cb8-4fc5-9746-d6060d49f9eb"). InnerVolumeSpecName "kube-api-access-ppnkq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:30:57.493166 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:57.493105 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ppnkq\" (UniqueName: \"kubernetes.io/projected/2bccc230-0cb8-4fc5-9746-d6060d49f9eb-kube-api-access-ppnkq\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:30:58.213065 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:58.213037 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" Jun 07 09:30:58.213245 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:58.213073 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680410-mm4ts" event={"ID":"2bccc230-0cb8-4fc5-9746-d6060d49f9eb","Type":"ContainerDied","Data":"1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab"} Jun 07 09:30:58.213245 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:30:58.213122 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1731e0e99bc0936050b804f25f6aa9b981eba2ef47f15e5ef46e58261a739cab" Jun 07 09:33:13.036499 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:33:13.036464 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:33:28.529987 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:33:28.529958 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:33:28.532439 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:33:28.532418 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:33:28.534355 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:33:28.534329 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:33:28.536870 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:33:28.536849 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:34:27.986943 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:34:27.986901 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:34:28.087637 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:34:28.087607 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:36:19.300876 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:36:19.300840 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:36:19.347999 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:36:19.347967 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:38:15.145682 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:15.145642 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:38:22.356061 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:22.356022 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:38:22.406446 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:22.406421 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:38:28.567151 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:28.567121 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:38:28.571044 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:28.571019 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:38:28.571590 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:28.571570 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:38:28.575644 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:38:28.575626 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:41:52.669725 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:41:52.669687 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:43:28.603768 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:43:28.603737 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:43:28.608266 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:43:28.608245 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:43:28.611627 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:43:28.611606 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:43:28.615660 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:43:28.615643 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t9kk8_0db1eeab-50d9-455d-800c-76e999b9995f/ovn-acl-logging/0.log" Jun 07 09:43:35.219088 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:43:35.219059 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:45:00.136480 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.136450 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29680425-mqgb2"] Jun 07 09:45:00.136973 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.136835 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2bccc230-0cb8-4fc5-9746-d6060d49f9eb" containerName="cleanup" Jun 07 09:45:00.136973 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.136848 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2bccc230-0cb8-4fc5-9746-d6060d49f9eb" containerName="cleanup" Jun 07 09:45:00.136973 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.136922 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2bccc230-0cb8-4fc5-9746-d6060d49f9eb" containerName="cleanup" Jun 07 09:45:00.139972 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.139956 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:00.142758 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.142732 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-9svnk\"" Jun 07 09:45:00.153821 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.153784 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29680425-mqgb2"] Jun 07 09:45:00.280582 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.280528 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmwff\" (UniqueName: \"kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff\") pod \"maas-api-key-cleanup-29680425-mqgb2\" (UID: \"d864b975-0499-4cd3-9a78-de7880c216cc\") " pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:00.382030 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.381989 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmwff\" (UniqueName: \"kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff\") pod \"maas-api-key-cleanup-29680425-mqgb2\" (UID: \"d864b975-0499-4cd3-9a78-de7880c216cc\") " pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:00.390961 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.390891 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmwff\" (UniqueName: \"kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff\") pod \"maas-api-key-cleanup-29680425-mqgb2\" (UID: \"d864b975-0499-4cd3-9a78-de7880c216cc\") " pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:00.450098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.450070 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:00.585763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.585715 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29680425-mqgb2"] Jun 07 09:45:00.593554 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:45:00.593525 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd864b975_0499_4cd3_9a78_de7880c216cc.slice/crio-65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395 WatchSource:0}: Error finding container 65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395: Status 404 returned error can't find the container with id 65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395 Jun 07 09:45:00.930578 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.930550 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" event={"ID":"d864b975-0499-4cd3-9a78-de7880c216cc","Type":"ContainerStarted","Data":"f62aec6cd903ac170434b5e8643f483d646b927011ba13910544406a0e9d3818"} Jun 07 09:45:00.930668 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.930598 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" event={"ID":"d864b975-0499-4cd3-9a78-de7880c216cc","Type":"ContainerStarted","Data":"65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395"} Jun 07 09:45:00.949209 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:00.949168 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" podStartSLOduration=0.94915348 podStartE2EDuration="949.15348ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-07 09:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:45:00.947049032 +0000 UTC m=+2192.948054155" watchObservedRunningTime="2026-06-07 09:45:00.94915348 +0000 UTC m=+2192.950158603" Jun 07 09:45:01.937518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:01.937484 2562 generic.go:356] "Generic (PLEG): container finished" podID="d864b975-0499-4cd3-9a78-de7880c216cc" containerID="f62aec6cd903ac170434b5e8643f483d646b927011ba13910544406a0e9d3818" exitCode=0 Jun 07 09:45:01.937913 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:01.937566 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" event={"ID":"d864b975-0499-4cd3-9a78-de7880c216cc","Type":"ContainerDied","Data":"f62aec6cd903ac170434b5e8643f483d646b927011ba13910544406a0e9d3818"} Jun 07 09:45:03.085748 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.085727 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:03.206723 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.206645 2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmwff\" (UniqueName: \"kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff\") pod \"d864b975-0499-4cd3-9a78-de7880c216cc\" (UID: \"d864b975-0499-4cd3-9a78-de7880c216cc\") " Jun 07 09:45:03.208650 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.208624 2562 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff" (OuterVolumeSpecName: "kube-api-access-bmwff") pod "d864b975-0499-4cd3-9a78-de7880c216cc" (UID: "d864b975-0499-4cd3-9a78-de7880c216cc"). InnerVolumeSpecName "kube-api-access-bmwff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 07 09:45:03.307293 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.307265 2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmwff\" (UniqueName: \"kubernetes.io/projected/d864b975-0499-4cd3-9a78-de7880c216cc-kube-api-access-bmwff\") on node \"ip-10-0-128-125.ec2.internal\" DevicePath \"\"" Jun 07 09:45:03.950271 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.950238 2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" Jun 07 09:45:03.950444 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.950288 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29680425-mqgb2" event={"ID":"d864b975-0499-4cd3-9a78-de7880c216cc","Type":"ContainerDied","Data":"65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395"} Jun 07 09:45:03.950444 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:03.950334 2562 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="65234a4d4ff2974d31f4b3164db15cf9baba6b0e61dd9e58c732b913649dd395" Jun 07 09:45:27.447054 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:27.447017 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:45:32.839494 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:32.839448 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:45:57.083129 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:45:57.083043 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:10.124950 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:10.124916 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:11.717970 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:11.717932 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:11.817225 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:11.817190 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:11.917514 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:11.917476 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:11.977892 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:11.977787 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:15.480553 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:15.480512 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:17.939113 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:17.939074 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:18.124934 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:18.124884 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:18.224999 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:18.224912 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:20.348453 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:20.348423 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:23.911863 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:23.911828 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:23.997061 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:23.997023 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:24.108277 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:24.108247 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:24.204262 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:24.204228 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:24.304658 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:24.304620 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:24.403577 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:24.403541 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:27.578548 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:27.578505 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:30.355604 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:30.355564 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:31.491563 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:31.491525 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:31.580371 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:31.576947 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:34.689678 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:34.689644 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:37.292388 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:37.292350 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:37.392763 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:37.392712 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:42.954467 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:42.954426 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:43.003712 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:43.003677 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:49.193518 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.193489 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-8f57c774b-vrg57_5a6ba7bb-ba43-4924-9b65-2347627cda5a/nginx/0.log" Jun 07 09:46:49.297991 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.297962 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-fvx8k_c9181244-b67c-4dde-acaf-2f99f0ae96bd/manager/0.log" Jun 07 09:46:49.415269 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.415243 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-cqmv5_6eb8ea7b-a00d-40bc-9862-1551322add25/manager/0.log" Jun 07 09:46:49.632431 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.632341 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29680410-mm4ts_2bccc230-0cb8-4fc5-9746-d6060d49f9eb/cleanup/0.log" Jun 07 09:46:49.739098 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.739072 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29680425-mqgb2_d864b975-0499-4cd3-9a78-de7880c216cc/cleanup/0.log" Jun 07 09:46:49.956768 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:49.956742 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-ztfgs_8e30405c-5d9b-4455-9104-59def15c6f19/server/0.log" Jun 07 09:46:50.079278 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:50.079249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-hw6j8_5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa/manager/0.log" Jun 07 09:46:50.320689 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:50.320605 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-7b88c_c2b95379-5adc-4ecf-a056-bc77dcb807d2/copy-manifests/0.log" Jun 07 09:46:50.326858 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:50.326833 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-7b88c_c2b95379-5adc-4ecf-a056-bc77dcb807d2/manager/0.log" Jun 07 09:46:50.540131 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:50.540083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-snbws_0a4a3251-c19a-468d-bffa-22112e7e5b72/postgres/0.log" Jun 07 09:46:51.269158 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.269101 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/extract/0.log" Jun 07 09:46:51.274099 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.274076 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/util/0.log" Jun 07 09:46:51.279331 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.279313 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/pull/0.log" Jun 07 09:46:51.382567 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.382539 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/util/0.log" Jun 07 09:46:51.387681 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.387661 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/pull/0.log" Jun 07 09:46:51.392600 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.392579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/extract/0.log" Jun 07 09:46:51.499350 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.499325 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/pull/0.log" Jun 07 09:46:51.505387 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.505360 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/extract/0.log" Jun 07 09:46:51.510593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.510572 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/util/0.log" Jun 07 09:46:51.617393 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.617330 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/extract/0.log" Jun 07 09:46:51.622394 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.622369 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/util/0.log" Jun 07 09:46:51.627725 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.627703 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/pull/0.log" Jun 07 09:46:51.992329 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:51.992305 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dvk8p_3e454ae3-ddd2-49e4-98cd-b409da965ef4/manager/0.log" Jun 07 09:46:53.025277 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:53.025251 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-ms5jr_a61902ac-e5ce-4728-bde1-3388deca1c93/discovery/0.log" Jun 07 09:46:53.127778 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:53.127754 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-gzprl_45ea3af8-7dfd-4d91-b6cc-700f89b4494a/kube-auth-proxy/0.log" Jun 07 09:46:53.352623 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:53.352543 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/1.log" Jun 07 09:46:53.678264 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:53.678234 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66bbd65f87-j4kdq_e897afd2-c08c-45a4-9466-f649cd07ef2f/router/0.log" Jun 07 09:46:54.241186 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:54.241160 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8_06908d4f-8e00-4244-8cb4-527e7fbfd98b/storage-initializer/0.log" Jun 07 09:46:54.248297 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:54.248278 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-68bb6c58d9-j9zp8_06908d4f-8e00-4244-8cb4-527e7fbfd98b/main/0.log" Jun 07 09:46:58.194916 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:58.194880 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:46:58.294501 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:46:58.294472 2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" limit=1024 Jun 07 09:47:01.141972 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:01.141939 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kvhzr_cb26a1c4-76fc-418a-b173-c34ed0071bc5/global-pull-secret-syncer/0.log" Jun 07 09:47:01.274176 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:01.274151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sfrqf_5c96ff84-adc4-4994-b119-12e61233577b/konnectivity-agent/0.log" Jun 07 09:47:01.363728 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:01.363697 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-125.ec2.internal_b0cb84e7292fcc17afc3acef5821ed4b/haproxy/0.log" Jun 07 09:47:04.170177 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:04.170151 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-operators_servicemesh-operator3-57f65f65fb-87pfd_4546ac4d-4d84-4794-a7b2-a8d1c3b74066/sail-operator/0.log" Jun 07 09:47:05.154540 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.154506 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/extract/0.log" Jun 07 09:47:05.200908 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.200877 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/util/0.log" Jun 07 09:47:05.227348 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.227320 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759kgjvc_363286c0-98be-4562-bed3-799d4b4d66fd/pull/0.log" Jun 07 09:47:05.255910 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.255888 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/extract/0.log" Jun 07 09:47:05.275871 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.275854 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/util/0.log" Jun 07 09:47:05.295703 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.295673 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e05gdcj_12deba9f-2cc7-4c42-94bb-fa3d39d72895/pull/0.log" Jun 07 09:47:05.320599 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.320579 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/extract/0.log" Jun 07 09:47:05.340200 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.340178 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/util/0.log" Jun 07 09:47:05.363008 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.362987 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738gc64_bc5f3dba-ac38-4c0f-b616-e523792285cb/pull/0.log" Jun 07 09:47:05.389605 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.389576 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/extract/0.log" Jun 07 09:47:05.409889 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.409824 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/util/0.log" Jun 07 09:47:05.433081 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.433064 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1ktv6l_06c188ec-9c16-4250-ba2b-ee5eb7ae3579/pull/0.log" Jun 07 09:47:05.689374 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:05.689349 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-dvk8p_3e454ae3-ddd2-49e4-98cd-b409da965ef4/manager/0.log" Jun 07 09:47:08.132165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:08.132083 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-85c9fd84d4-lltd4_2c6a610e-a35b-4c71-8d9c-80ba2f38a9cd/metrics-server/0.log" Jun 07 09:47:08.160705 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:08.160676 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-76b68cc874-d8tg6_d1611ab7-07a0-4f5b-a326-ff62f2ed2caf/monitoring-plugin/0.log" Jun 07 09:47:08.434438 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:08.434411 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz66r_7854433d-36c6-40a9-8c5f-26bfed42b8d2/node-exporter/0.log" Jun 07 09:47:08.460165 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:08.460139 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz66r_7854433d-36c6-40a9-8c5f-26bfed42b8d2/kube-rbac-proxy/0.log" Jun 07 09:47:08.482856 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:08.482838 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-tz66r_7854433d-36c6-40a9-8c5f-26bfed42b8d2/init-textfile/0.log" Jun 07 09:47:10.842969 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:10.842939 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d5d5bb78b-zmsvt_8afb1a67-475c-4eb6-95b9-0022fad8cc23/networking-console-plugin/0.log" Jun 07 09:47:11.459298 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:11.459267 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/1.log" Jun 07 09:47:11.469460 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:11.469431 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-668fdbf85c-hfnrb_9781214e-f19d-4d43-a164-156e453a26a8/console-operator/2.log" Jun 07 09:47:13.479627 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.479598 2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8"] Jun 07 09:47:13.480013 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.479954 2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d864b975-0499-4cd3-9a78-de7880c216cc" containerName="cleanup" Jun 07 09:47:13.480013 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.479965 2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d864b975-0499-4cd3-9a78-de7880c216cc" containerName="cleanup" Jun 07 09:47:13.480087 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.480049 2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d864b975-0499-4cd3-9a78-de7880c216cc" containerName="cleanup" Jun 07 09:47:13.483117 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.483099 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.487347 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.487319 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfwxs\"/\"openshift-service-ca.crt\"" Jun 07 09:47:13.487489 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.487358 2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jfwxs\"/\"kube-root-ca.crt\"" Jun 07 09:47:13.487489 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.487319 2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jfwxs\"/\"default-dockercfg-nh8jc\"" Jun 07 09:47:13.491461 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.491442 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8"] Jun 07 09:47:13.605468 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.605437 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-lib-modules\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.605630 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.605480 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-sys\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.605630 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.605502 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-podres\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.605630 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.605602 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkp7q\" (UniqueName: \"kubernetes.io/projected/87c17f8c-38ba-4323-bad0-e1c584728171-kube-api-access-tkp7q\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.605754 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.605666 2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-proc\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706204 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706172 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-lib-modules\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706221 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-sys\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706250 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-podres\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706303 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkp7q\" (UniqueName: \"kubernetes.io/projected/87c17f8c-38ba-4323-bad0-e1c584728171-kube-api-access-tkp7q\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706351 2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-proc\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706367 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-lib-modules\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706367 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-sys\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706452 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706410 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-podres\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.706728 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.706455 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/87c17f8c-38ba-4323-bad0-e1c584728171-proc\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.715105 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.715083 2562 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkp7q\" (UniqueName: \"kubernetes.io/projected/87c17f8c-38ba-4323-bad0-e1c584728171-kube-api-access-tkp7q\") pod \"perf-node-gather-daemonset-nt4t8\" (UID: \"87c17f8c-38ba-4323-bad0-e1c584728171\") " pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.794385 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.794318 2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:13.922622 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.922596 2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8"] Jun 07 09:47:13.923901 ip-10-0-128-125 kubenswrapper[2562]: W0607 09:47:13.923874 2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod87c17f8c_38ba_4323_bad0_e1c584728171.slice/crio-de3b716e142b096e00dff0b1d56ed2671bd078002a2dade5b23cf9f9bc586d85 WatchSource:0}: Error finding container de3b716e142b096e00dff0b1d56ed2671bd078002a2dade5b23cf9f9bc586d85: Status 404 returned error can't find the container with id de3b716e142b096e00dff0b1d56ed2671bd078002a2dade5b23cf9f9bc586d85 Jun 07 09:47:13.971404 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.971385 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9pv2c_50a6ee97-e3e3-489c-b2f8-0f20de1fdd57/dns/0.log" Jun 07 09:47:13.990593 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:13.990575 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-9pv2c_50a6ee97-e3e3-489c-b2f8-0f20de1fdd57/kube-rbac-proxy/0.log" Jun 07 09:47:14.149761 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.149676 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vrbp2_91483ade-61fa-4480-8667-b5f93faa14c4/dns-node-resolver/0.log" Jun 07 09:47:14.662359 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.662328 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56d8457687-n48gs_1727d6ab-acde-42bc-b86f-0e98d5949119/registry/0.log" Jun 07 09:47:14.677381 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.677356 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-56d8457687-n48gs_1727d6ab-acde-42bc-b86f-0e98d5949119/registry/1.log" Jun 07 09:47:14.693631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.693604 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" event={"ID":"87c17f8c-38ba-4323-bad0-e1c584728171","Type":"ContainerStarted","Data":"7e8dd2809c3cb4158d9e7751311dd6485ba5a8521475e6b453197e0c3d995777"} Jun 07 09:47:14.693760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.693652 2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" event={"ID":"87c17f8c-38ba-4323-bad0-e1c584728171","Type":"ContainerStarted","Data":"de3b716e142b096e00dff0b1d56ed2671bd078002a2dade5b23cf9f9bc586d85"} Jun 07 09:47:14.693760 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.693753 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:14.698299 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.698281 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-j8gj8_875807c0-7e1f-4b5d-b2e9-7a5665aa0e07/node-ca/0.log" Jun 07 09:47:14.709550 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:14.709512 2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" podStartSLOduration=1.7094960719999999 podStartE2EDuration="1.709496072s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-07 09:47:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-07 09:47:14.708976081 +0000 UTC m=+2326.709981215" watchObservedRunningTime="2026-06-07 09:47:14.709496072 +0000 UTC m=+2326.710501194" Jun 07 09:47:15.778004 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:15.777969 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-ms5jr_a61902ac-e5ce-4728-bde1-3388deca1c93/discovery/0.log" Jun 07 09:47:15.798415 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:15.798390 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-gzprl_45ea3af8-7dfd-4d91-b6cc-700f89b4494a/kube-auth-proxy/0.log" Jun 07 09:47:15.847008 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:15.846985 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/0.log" Jun 07 09:47:15.910838 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:15.910788 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/1.log" Jun 07 09:47:16.031713 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:16.031638 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66bbd65f87-j4kdq_e897afd2-c08c-45a4-9466-f649cd07ef2f/router/0.log" Jun 07 09:47:16.627486 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:16.627458 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4z8nf_7b820800-1a37-4938-9cd6-0cb826052e09/serve-healthcheck-canary/0.log" Jun 07 09:47:17.209464 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:17.209437 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj54p_3e48adb9-4c2c-4f3c-ad6d-20029700f62a/kube-rbac-proxy/0.log" Jun 07 09:47:17.227773 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:17.227753 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj54p_3e48adb9-4c2c-4f3c-ad6d-20029700f62a/exporter/0.log" Jun 07 09:47:17.247685 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:17.247667 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dj54p_3e48adb9-4c2c-4f3c-ad6d-20029700f62a/extractor/0.log" Jun 07 09:47:19.509911 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.509875 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-8f57c774b-vrg57_5a6ba7bb-ba43-4924-9b65-2347627cda5a/nginx/0.log" Jun 07 09:47:19.531647 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.531622 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-69fc866d76-fvx8k_c9181244-b67c-4dde-acaf-2f99f0ae96bd/manager/0.log" Jun 07 09:47:19.679619 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.679581 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-864bfd7bc9-cqmv5_6eb8ea7b-a00d-40bc-9862-1551322add25/manager/0.log" Jun 07 09:47:19.731825 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.731784 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29680410-mm4ts_2bccc230-0cb8-4fc5-9746-d6060d49f9eb/cleanup/0.log" Jun 07 09:47:19.751946 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.751915 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29680425-mqgb2_d864b975-0499-4cd3-9a78-de7880c216cc/cleanup/0.log" Jun 07 09:47:19.845176 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.845088 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-5dd488fc48-ztfgs_8e30405c-5d9b-4455-9104-59def15c6f19/server/0.log" Jun 07 09:47:19.915471 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:19.915444 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-6dc94f689f-hw6j8_5cf5f7a6-5ebf-4ffa-b692-1774e39dc8fa/manager/0.log" Jun 07 09:47:20.152553 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:20.152468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-7b88c_c2b95379-5adc-4ecf-a056-bc77dcb807d2/manager/0.log" Jun 07 09:47:20.170514 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:20.170468 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c76964bf-7b88c_c2b95379-5adc-4ecf-a056-bc77dcb807d2/copy-manifests/0.log" Jun 07 09:47:20.233884 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:20.233849 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-snbws_0a4a3251-c19a-468d-bffa-22112e7e5b72/postgres/0.log" Jun 07 09:47:20.711139 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:20.711113 2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jfwxs/perf-node-gather-daemonset-nt4t8" Jun 07 09:47:21.756516 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:21.756472 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-njt88_af803b05-4240-4341-8bb9-087c399cedb7/openshift-lws-operator/0.log" Jun 07 09:47:26.279173 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:26.279132 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-ms5jr_a61902ac-e5ce-4728-bde1-3388deca1c93/discovery/0.log" Jun 07 09:47:26.288502 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:26.288474 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-gzprl_45ea3af8-7dfd-4d91-b6cc-700f89b4494a/kube-auth-proxy/0.log" Jun 07 09:47:26.326095 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:26.326064 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/0.log" Jun 07 09:47:26.388514 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:26.388491 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/1.log" Jun 07 09:47:26.495631 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:26.495602 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66bbd65f87-j4kdq_e897afd2-c08c-45a4-9466-f649cd07ef2f/router/0.log" Jun 07 09:47:27.658520 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:27.658490 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-ms5jr_a61902ac-e5ce-4728-bde1-3388deca1c93/discovery/0.log" Jun 07 09:47:27.669986 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:27.669965 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-gzprl_45ea3af8-7dfd-4d91-b6cc-700f89b4494a/kube-auth-proxy/0.log" Jun 07 09:47:27.707983 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:27.707957 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/0.log" Jun 07 09:47:27.773988 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:27.773945 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/1.log" Jun 07 09:47:27.879994 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:27.879966 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66bbd65f87-j4kdq_e897afd2-c08c-45a4-9466-f649cd07ef2f/router/0.log" Jun 07 09:47:29.217695 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.217665 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/kube-multus-additional-cni-plugins/0.log" Jun 07 09:47:29.239195 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.239170 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/egress-router-binary-copy/0.log" Jun 07 09:47:29.259682 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.259661 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/cni-plugins/0.log" Jun 07 09:47:29.278195 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.278175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/bond-cni-plugin/0.log" Jun 07 09:47:29.296269 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.296251 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/routeoverride-cni/0.log" Jun 07 09:47:29.314875 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.314853 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/whereabouts-cni-bincopy/0.log" Jun 07 09:47:29.333275 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.333249 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tfhnf_0e1f3537-4417-495d-ada4-ca6c7a9c3ad1/whereabouts-cni/0.log" Jun 07 09:47:29.404895 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.404870 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gmbth_64765493-ea44-4117-a576-f301ec7e4df1/kube-multus/0.log" Jun 07 09:47:29.543865 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.543773 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rkp7x_d72371b8-2fcd-44d3-8280-1b3af902f221/network-metrics-daemon/0.log" Jun 07 09:47:29.560112 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.560084 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rkp7x_d72371b8-2fcd-44d3-8280-1b3af902f221/kube-rbac-proxy/0.log" Jun 07 09:47:29.691524 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.691494 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-75c67f8887-ms5jr_a61902ac-e5ce-4728-bde1-3388deca1c93/discovery/0.log" Jun 07 09:47:29.700134 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.700107 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-f78cc86d8-gzprl_45ea3af8-7dfd-4d91-b6cc-700f89b4494a/kube-auth-proxy/0.log" Jun 07 09:47:29.731473 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.731434 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/0.log" Jun 07 09:47:29.795881 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.795757 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-687ff6996-6lzr2_bd7864dc-3cd8-4dd5-a5cd-2d682fd18a49/istio-proxy/1.log" Jun 07 09:47:29.891205 ip-10-0-128-125 kubenswrapper[2562]: I0607 09:47:29.891175 2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-66bbd65f87-j4kdq_e897afd2-c08c-45a4-9466-f649cd07ef2f/router/0.log"