Jun 12 17:08:50.653376 ip-10-0-128-121 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 12 17:08:50.653391 ip-10-0-128-121 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 12 17:08:50.653400 ip-10-0-128-121 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 12 17:08:50.653772 ip-10-0-128-121 systemd[1]: Failed to start Kubernetes Kubelet. Jun 12 17:09:00.669321 ip-10-0-128-121 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 12 17:09:00.669343 ip-10-0-128-121 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f6b5a3dcadbb462a9b82cf6f615df1da -- Jun 12 17:11:23.003622 ip-10-0-128-121 systemd[1]: Starting Kubernetes Kubelet... Jun 12 17:11:23.424981 ip-10-0-128-121 kubenswrapper[2544]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 17:11:23.424981 ip-10-0-128-121 kubenswrapper[2544]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 12 17:11:23.424981 ip-10-0-128-121 kubenswrapper[2544]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 17:11:23.424981 ip-10-0-128-121 kubenswrapper[2544]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 12 17:11:23.424981 ip-10-0-128-121 kubenswrapper[2544]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 12 17:11:23.425673 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.425047 2544 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 12 17:11:23.428723 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428704 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.428723 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428721 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.428723 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428725 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428728 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428732 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428736 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428739 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428742 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428745 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428748 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428750 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428753 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428755 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428758 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428768 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428771 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428773 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428776 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428778 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428781 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428783 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.428816 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428786 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428790 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428794 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428797 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428800 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428802 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428805 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428808 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428810 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428813 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428815 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428817 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428820 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428823 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428825 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428830 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428834 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428837 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428840 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.429264 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428842 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428845 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428848 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428850 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428853 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428857 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428859 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428861 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428864 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428866 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428869 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428871 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428874 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428876 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428879 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428884 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428886 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428889 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428892 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428894 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.429734 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428897 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428899 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428902 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428905 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428907 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428910 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428912 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428915 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428917 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428921 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428923 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428926 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428929 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428931 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428934 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428936 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428939 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428942 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428944 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428947 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.430224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428949 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428952 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428954 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428957 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428959 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428961 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428964 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428968 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428972 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428975 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428978 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428981 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428983 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428986 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428989 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428991 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428993 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428996 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.430762 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.428999 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.432374 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432360 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.432374 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432373 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432377 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432380 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432383 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432387 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432390 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432393 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432395 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432398 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432401 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432404 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432407 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432411 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432413 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432416 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432419 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432421 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432424 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432426 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432429 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.432445 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432431 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432434 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432436 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432439 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432441 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432444 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432446 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432449 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432452 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432454 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432458 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432460 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432463 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432465 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432468 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432471 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432473 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432476 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432478 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432481 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.432941 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432483 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432485 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432488 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432490 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432493 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432495 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432498 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432500 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432503 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432505 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432509 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432512 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432514 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432517 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432520 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432522 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432526 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432531 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432534 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.433431 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432538 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432541 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432543 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432546 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432549 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432552 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432555 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432557 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432560 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432562 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432565 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432567 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432570 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432572 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432575 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432577 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432580 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432585 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432587 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.433899 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432590 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432593 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432596 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432598 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432601 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432604 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432606 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432609 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432612 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432614 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432617 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432620 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432622 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432625 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432627 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432630 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432632 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432635 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432638 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.432640 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.434401 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433068 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433073 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433076 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433079 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433081 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433084 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433086 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433089 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433092 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433094 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433097 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433102 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433105 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433108 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433111 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433114 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433118 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433122 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433125 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.434883 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433132 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433135 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433137 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433140 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433143 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433146 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433149 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433152 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433154 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433157 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433160 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433163 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433166 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433168 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433171 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433174 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433176 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433178 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433181 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433183 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.435365 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433186 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433188 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433191 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433193 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433196 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433199 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433201 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433203 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433206 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433210 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433212 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433215 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433219 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433222 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433224 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433227 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433229 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433232 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433235 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433237 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.435839 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433240 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433242 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433247 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433251 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433253 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433256 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433259 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433261 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433264 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433266 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433269 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433272 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433274 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433277 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433280 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433283 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433285 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433288 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433305 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.436383 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433308 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433311 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433314 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433316 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433319 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433324 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433326 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433330 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433333 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433336 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433339 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433341 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433344 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433346 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433349 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433352 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433354 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433357 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433360 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.436841 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433362 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.433365 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433447 2544 flags.go:64] FLAG: --address="0.0.0.0" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433454 2544 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433460 2544 flags.go:64] FLAG: --anonymous-auth="true" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433465 2544 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433469 2544 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433473 2544 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433487 2544 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433492 2544 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433495 2544 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433498 2544 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433502 2544 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433506 2544 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433509 2544 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433512 2544 flags.go:64] FLAG: --cgroup-root="" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433515 2544 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433518 2544 flags.go:64] FLAG: --client-ca-file="" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433522 2544 flags.go:64] FLAG: --cloud-provider="external" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433525 2544 flags.go:64] FLAG: --cluster-dns="[]" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433529 2544 flags.go:64] FLAG: --cluster-domain="" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433532 2544 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433535 2544 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433538 2544 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 12 17:11:23.437313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433541 2544 flags.go:64] FLAG: --container-log-max-files="5" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433545 2544 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433548 2544 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433551 2544 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433554 2544 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433557 2544 flags.go:64] FLAG: --contention-profiling="false" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433560 2544 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433563 2544 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433567 2544 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433569 2544 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433574 2544 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433576 2544 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433579 2544 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433582 2544 flags.go:64] FLAG: --enable-load-reader="false" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433585 2544 flags.go:64] FLAG: --enable-server="true" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433588 2544 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433592 2544 flags.go:64] FLAG: --event-burst="100" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433595 2544 flags.go:64] FLAG: --event-qps="50" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433598 2544 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433602 2544 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433605 2544 flags.go:64] FLAG: --eviction-hard="" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433609 2544 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433612 2544 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433616 2544 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433619 2544 flags.go:64] FLAG: --eviction-soft="" Jun 12 17:11:23.437890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433622 2544 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433627 2544 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433629 2544 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433632 2544 flags.go:64] FLAG: --experimental-mounter-path="" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433635 2544 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433638 2544 flags.go:64] FLAG: --fail-swap-on="true" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433640 2544 flags.go:64] FLAG: --feature-gates="" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433644 2544 flags.go:64] FLAG: --file-check-frequency="20s" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433647 2544 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433651 2544 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433653 2544 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433656 2544 flags.go:64] FLAG: --healthz-port="10248" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433660 2544 flags.go:64] FLAG: --help="false" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433663 2544 flags.go:64] FLAG: --hostname-override="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433666 2544 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433669 2544 flags.go:64] FLAG: --http-check-frequency="20s" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433672 2544 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433675 2544 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433679 2544 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433682 2544 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433684 2544 flags.go:64] FLAG: --image-service-endpoint="" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433687 2544 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433690 2544 flags.go:64] FLAG: --kube-api-burst="100" Jun 12 17:11:23.438548 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433693 2544 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433696 2544 flags.go:64] FLAG: --kube-api-qps="50" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433699 2544 flags.go:64] FLAG: --kube-reserved="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433702 2544 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433704 2544 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433707 2544 flags.go:64] FLAG: --kubelet-cgroups="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433711 2544 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433715 2544 flags.go:64] FLAG: --lock-file="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433718 2544 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433721 2544 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433725 2544 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433731 2544 flags.go:64] FLAG: --log-json-split-stream="false" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433734 2544 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433736 2544 flags.go:64] FLAG: --log-text-split-stream="false" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433739 2544 flags.go:64] FLAG: --logging-format="text" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433742 2544 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433745 2544 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433748 2544 flags.go:64] FLAG: --manifest-url="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433751 2544 flags.go:64] FLAG: --manifest-url-header="" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433755 2544 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433758 2544 flags.go:64] FLAG: --max-open-files="1000000" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433763 2544 flags.go:64] FLAG: --max-pods="110" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433769 2544 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433772 2544 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433775 2544 flags.go:64] FLAG: --memory-manager-policy="None" Jun 12 17:11:23.439127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433778 2544 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433780 2544 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433783 2544 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433786 2544 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433794 2544 flags.go:64] FLAG: --node-status-max-images="50" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433797 2544 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433800 2544 flags.go:64] FLAG: --oom-score-adj="-999" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433803 2544 flags.go:64] FLAG: --pod-cidr="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433806 2544 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4a2e90c8ef7d783e4d635e24fb09435f5e1f6e954583512c0944f9e57bc77b52" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433812 2544 flags.go:64] FLAG: --pod-manifest-path="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433815 2544 flags.go:64] FLAG: --pod-max-pids="-1" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433818 2544 flags.go:64] FLAG: --pods-per-core="0" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433821 2544 flags.go:64] FLAG: --port="10250" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433824 2544 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433827 2544 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0967eae89f2e9b797" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433832 2544 flags.go:64] FLAG: --qos-reserved="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433835 2544 flags.go:64] FLAG: --read-only-port="10255" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433839 2544 flags.go:64] FLAG: --register-node="true" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433842 2544 flags.go:64] FLAG: --register-with-taints="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433846 2544 flags.go:64] FLAG: --registry-burst="10" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433848 2544 flags.go:64] FLAG: --registry-qps="5" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433851 2544 flags.go:64] FLAG: --reserved-cpus="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433854 2544 flags.go:64] FLAG: --reserved-memory="" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433857 2544 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433861 2544 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 12 17:11:23.439794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433863 2544 flags.go:64] FLAG: --rotate-certificates="false" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433866 2544 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433869 2544 flags.go:64] FLAG: --runonce="false" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433871 2544 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433874 2544 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433877 2544 flags.go:64] FLAG: --seccomp-default="false" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433880 2544 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433883 2544 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433885 2544 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433888 2544 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433892 2544 flags.go:64] FLAG: --storage-driver-password="root" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433895 2544 flags.go:64] FLAG: --storage-driver-secure="false" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433898 2544 flags.go:64] FLAG: --storage-driver-table="stats" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433901 2544 flags.go:64] FLAG: --storage-driver-user="root" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433904 2544 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433907 2544 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433910 2544 flags.go:64] FLAG: --system-cgroups="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433913 2544 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433918 2544 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433921 2544 flags.go:64] FLAG: --tls-cert-file="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433923 2544 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433927 2544 flags.go:64] FLAG: --tls-min-version="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433932 2544 flags.go:64] FLAG: --tls-private-key-file="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433935 2544 flags.go:64] FLAG: --topology-manager-policy="none" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433939 2544 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 12 17:11:23.440521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433942 2544 flags.go:64] FLAG: --topology-manager-scope="container" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433945 2544 flags.go:64] FLAG: --v="2" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433949 2544 flags.go:64] FLAG: --version="false" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433953 2544 flags.go:64] FLAG: --vmodule="" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433957 2544 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.433960 2544 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434092 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434096 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434098 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434102 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434104 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434107 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434110 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434112 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434115 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434119 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434123 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434126 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434128 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434131 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434134 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.441144 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434137 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434139 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434141 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434144 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434146 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434149 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434151 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434154 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434158 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434161 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434165 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434168 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434170 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434173 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434192 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434196 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434200 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434202 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434205 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434208 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.441664 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434211 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434213 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434216 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434219 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434222 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434225 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434228 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434231 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434234 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434237 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434240 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434243 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434245 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434248 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434250 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434254 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434257 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434260 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.442159 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434262 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434265 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434269 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434271 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434277 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434279 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434282 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434284 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434287 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434304 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434307 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434310 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434312 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434314 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434317 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434319 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434322 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434324 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434327 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434329 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.442675 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434332 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434334 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434337 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434339 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434342 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434344 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434346 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434349 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434351 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434354 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434356 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434359 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434361 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434363 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434367 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434370 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434373 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434376 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434379 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434381 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.443155 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434384 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434386 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434389 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434391 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434394 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.434396 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.435859 2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.442944 2544 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.442965 2544 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.442982 2544 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.442993 2544 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443066 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443071 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443074 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443077 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.443665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443080 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443083 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443086 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443089 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443092 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443094 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443097 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443100 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443102 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443107 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443111 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443114 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443117 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443121 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443124 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443126 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443129 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443131 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443134 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.444047 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443137 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443140 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443142 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443145 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443147 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443150 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443157 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443160 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443163 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443166 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443168 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443171 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443173 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443176 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443178 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443181 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443183 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443186 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443188 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443191 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.444513 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443193 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443195 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443198 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443200 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443202 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443205 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443208 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443211 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443214 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443216 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443218 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443221 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443224 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443226 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443230 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443233 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443236 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443239 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443241 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443245 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.444989 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443247 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443250 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443252 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443255 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443257 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443260 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443262 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443265 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443267 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443270 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443272 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443275 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443277 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443279 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443282 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443284 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443287 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443308 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443312 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.445515 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443315 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443317 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443320 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443322 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443325 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443328 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443330 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443333 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443335 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443338 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443340 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443343 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443345 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443349 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443353 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443357 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.445964 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443360 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.443365 2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443487 2544 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443492 2544 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443495 2544 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443498 2544 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443501 2544 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443504 2544 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443506 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443509 2544 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443514 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443517 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443519 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443522 2544 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443524 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 12 17:11:23.446363 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443527 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443530 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443532 2544 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443535 2544 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443537 2544 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443540 2544 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443543 2544 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443545 2544 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443548 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443550 2544 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443552 2544 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443555 2544 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443557 2544 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443560 2544 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443562 2544 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443565 2544 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443568 2544 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443570 2544 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443573 2544 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443575 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 12 17:11:23.446729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443577 2544 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443580 2544 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443582 2544 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443585 2544 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443587 2544 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443590 2544 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443592 2544 feature_gate.go:328] unrecognized feature gate: Example Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443594 2544 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443597 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443600 2544 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443603 2544 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443605 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443608 2544 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443610 2544 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443613 2544 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443615 2544 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443617 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443620 2544 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443622 2544 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443624 2544 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 12 17:11:23.447221 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443627 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443629 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443632 2544 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443634 2544 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443637 2544 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443639 2544 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443642 2544 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443644 2544 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443647 2544 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443649 2544 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443651 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443654 2544 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443656 2544 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443659 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443661 2544 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443664 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443667 2544 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443669 2544 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443672 2544 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 12 17:11:23.447791 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443674 2544 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443680 2544 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443684 2544 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443687 2544 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443691 2544 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443694 2544 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443696 2544 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443699 2544 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443701 2544 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443704 2544 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443706 2544 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443709 2544 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443711 2544 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443714 2544 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443716 2544 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443719 2544 feature_gate.go:328] unrecognized feature gate: EventTTL Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443723 2544 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443726 2544 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443728 2544 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jun 12 17:11:23.448260 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443731 2544 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443734 2544 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443736 2544 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443739 2544 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443741 2544 feature_gate.go:328] unrecognized feature gate: OSStreams Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443744 2544 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443747 2544 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:23.443749 2544 feature_gate.go:328] unrecognized feature gate: Example2 Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.443754 2544 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.443883 2544 server.go:962] "Client rotation is on, will bootstrap in background" Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.446012 2544 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.447376 2544 server.go:1019] "Starting client certificate rotation" Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.447483 2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 12 17:11:23.448727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.447552 2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 12 17:11:23.452717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.452689 2544 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 12 17:11:23.457053 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.457034 2544 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 12 17:11:23.478815 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.478783 2544 log.go:25] "Validated CRI v1 runtime API" Jun 12 17:11:23.482627 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.482600 2544 log.go:25] "Validated CRI v1 image API" Jun 12 17:11:23.484012 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.483964 2544 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 12 17:11:23.486621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.486597 2544 fs.go:135] Filesystem UUIDs: map[5e7393a8-ee6e-45ac-addc-a29e364220c6:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9717df68-81dd-4597-a8c3-62215f41d068:/dev/nvme0n1p4] Jun 12 17:11:23.486621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.486619 2544 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 12 17:11:23.490317 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.490285 2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 12 17:11:23.493203 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.493078 2544 manager.go:216] Machine: {Timestamp:2026-06-12 17:11:23.491638248 +0000 UTC m=+0.364286368 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100983 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22a087b90a420821494ce4c92cbf1e SystemUUID:ec22a087-b90a-4208-2149-4ce4c92cbf1e BootID:f6b5a3dc-adbb-462a-9b82-cf6f615df1da Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8c:31:dd:5e:eb Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8c:31:dd:5e:eb Speed:0 Mtu:9001} {Name:ovs-system MacAddress:22:47:93:22:b3:2d Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 12 17:11:23.493203 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.493193 2544 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 12 17:11:23.493369 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.493356 2544 manager.go:232] Version: {KernelVersion:5.14.0-570.120.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260608-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 12 17:11:23.496335 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.496284 2544 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 12 17:11:23.496499 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.496338 2544 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-121.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 12 17:11:23.497014 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.497004 2544 topology_manager.go:138] "Creating topology manager with none policy" Jun 12 17:11:23.497049 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.497017 2544 container_manager_linux.go:309] "Creating device plugin manager" Jun 12 17:11:23.497049 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.497025 2544 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 12 17:11:23.497684 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.497673 2544 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 12 17:11:23.498973 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.498961 2544 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jun 12 17:11:23.501846 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.501830 2544 state_mem.go:36] "Initialized new in-memory state store" Jun 12 17:11:23.501995 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.501986 2544 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jun 12 17:11:23.502035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502019 2544 kubelet.go:486] "Attempting to sync node with API server" Jun 12 17:11:23.502035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502031 2544 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 12 17:11:23.502090 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502041 2544 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 12 17:11:23.502090 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502051 2544 kubelet.go:398] "Adding apiserver pod source" Jun 12 17:11:23.502090 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502063 2544 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 12 17:11:23.502507 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502493 2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 12 17:11:23.502564 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.502519 2544 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 12 17:11:23.506766 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.506721 2544 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jun 12 17:11:23.509776 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.509756 2544 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 12 17:11:23.509879 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.509795 2544 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jun 12 17:11:23.509936 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.509888 2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-45rps" Jun 12 17:11:23.510465 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510450 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510470 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510479 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510487 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510499 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510507 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510519 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 12 17:11:23.510533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510527 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 12 17:11:23.510748 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510538 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 12 17:11:23.510748 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510547 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 12 17:11:23.510748 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510560 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 12 17:11:23.510748 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.510568 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 12 17:11:23.511264 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.511253 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 12 17:11:23.511345 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.511267 2544 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 12 17:11:23.513673 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.513651 2544 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-121.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 12 17:11:23.514383 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.514354 2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-121.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 12 17:11:23.514462 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.514357 2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 12 17:11:23.514981 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.514966 2544 server.go:1268] "Started kubelet" Jun 12 17:11:23.515050 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.515019 2544 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 12 17:11:23.515110 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.515053 2544 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 12 17:11:23.515173 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.515155 2544 server_v1.go:49] "podresources" method="list" useActivePods=true Jun 12 17:11:23.515982 ip-10-0-128-121 systemd[1]: Started Kubernetes Kubelet. Jun 12 17:11:23.516876 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.516867 2544 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 12 17:11:23.517402 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.517387 2544 server.go:310] "Adding debug handlers to kubelet server" Jun 12 17:11:23.518178 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.517988 2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-45rps" Jun 12 17:11:23.520035 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.519012 2544 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-121.ec2.internal.18b8649e42e4eaba default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-121.ec2.internal,UID:ip-10-0-128-121.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-121.ec2.internal,},FirstTimestamp:2026-06-12 17:11:23.51494009 +0000 UTC m=+0.387588190,LastTimestamp:2026-06-12 17:11:23.51494009 +0000 UTC m=+0.387588190,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-121.ec2.internal,}" Jun 12 17:11:23.522487 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.522466 2544 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 12 17:11:23.522652 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.522636 2544 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 12 17:11:23.522746 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.522732 2544 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 12 17:11:23.522802 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.522775 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.523372 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523350 2544 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jun 12 17:11:23.523480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523354 2544 volume_manager.go:311] "The desired_state_of_world populator starts" Jun 12 17:11:23.523480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523431 2544 volume_manager.go:313] "Starting Kubelet Volume Manager" Jun 12 17:11:23.523579 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523530 2544 reconstruct.go:97] "Volume reconstruction finished" Jun 12 17:11:23.523579 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523540 2544 reconciler.go:29] "Reconciler: start to sync state" Jun 12 17:11:23.523691 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523612 2544 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 12 17:11:23.523691 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523622 2544 factory.go:55] Registering systemd factory Jun 12 17:11:23.523691 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523629 2544 factory.go:223] Registration of the systemd container factory successfully Jun 12 17:11:23.523950 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523933 2544 factory.go:153] Registering CRI-O factory Jun 12 17:11:23.524035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523952 2544 factory.go:223] Registration of the crio container factory successfully Jun 12 17:11:23.524035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.523976 2544 factory.go:103] Registering Raw factory Jun 12 17:11:23.524035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.524001 2544 manager.go:1196] Started watching for new ooms in manager Jun 12 17:11:23.524382 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.524367 2544 manager.go:349] Starting recovery of all containers Jun 12 17:11:23.533603 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.533576 2544 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:23.534142 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.534119 2544 manager.go:354] Recovery completed Jun 12 17:11:23.537330 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.537283 2544 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-121.ec2.internal\" not found" node="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.539753 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.539736 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.541935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.541912 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.542014 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.541957 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.542014 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.541972 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.542571 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.542557 2544 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 12 17:11:23.542623 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.542570 2544 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 12 17:11:23.542623 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.542588 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.542699 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.542592 2544 state_mem.go:36] "Initialized new in-memory state store" Jun 12 17:11:23.543346 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543330 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.543403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543361 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.543403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543376 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.543892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543881 2544 policy_none.go:49] "None policy: Start" Jun 12 17:11:23.543931 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543897 2544 memory_manager.go:187] "Starting memorymanager" policy="None" Jun 12 17:11:23.543931 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.543908 2544 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jun 12 17:11:23.544915 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.544905 2544 policy_none.go:47] "Start" Jun 12 17:11:23.582034 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582012 2544 manager.go:337] "Starting Device Plugin manager" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.582065 2544 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582081 2544 server.go:87] "Starting device plugin registration server" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582398 2544 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582413 2544 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582498 2544 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582597 2544 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.582604 2544 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.583225 2544 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.583279 2544 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.595243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.594966 2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jun 12 17:11:23.596427 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.596410 2544 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jun 12 17:11:23.596478 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.596454 2544 status_manager.go:244] "Starting to sync pod status with apiserver" Jun 12 17:11:23.596531 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.596480 2544 kubelet.go:2448] "Starting kubelet main sync loop" Jun 12 17:11:23.596531 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.596521 2544 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 12 17:11:23.599229 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.599202 2544 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:23.682958 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.682845 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.683802 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.683785 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.683918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.683823 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.683918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.683840 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.683918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.683870 2544 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.692807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.692781 2544 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.692904 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.692812 2544 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-121.ec2.internal\": node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.696578 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.696555 2544 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal"] Jun 12 17:11:23.696681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.696622 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.697538 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.697520 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.697622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.697551 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.697622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.697561 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.698912 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.698900 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.699055 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.699026 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.699109 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.699065 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.704567 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.704543 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.704988 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.704970 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.705088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.705006 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.705088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.705019 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.705088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.704974 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.705088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.705062 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.705088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.705073 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.706263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.706244 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.706392 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.706272 2544 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jun 12 17:11:23.707024 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.707007 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientMemory" Jun 12 17:11:23.707097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.707035 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasNoDiskPressure" Jun 12 17:11:23.707097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.707050 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeHasSufficientPID" Jun 12 17:11:23.728351 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.728318 2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-121.ec2.internal\" not found" node="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.733204 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.733185 2544 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-121.ec2.internal\" not found" node="ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.805371 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.805341 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.824556 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.824533 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d7b965d585d2a6f0e3ab4a9c481d7616-config\") pod \"kube-apiserver-proxy-ip-10-0-128-121.ec2.internal\" (UID: \"d7b965d585d2a6f0e3ab4a9c481d7616\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.824662 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.824564 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.824662 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.824582 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.905903 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:23.905863 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:23.925284 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925259 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.925400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925287 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.925400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925321 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d7b965d585d2a6f0e3ab4a9c481d7616-config\") pod \"kube-apiserver-proxy-ip-10-0-128-121.ec2.internal\" (UID: \"d7b965d585d2a6f0e3ab4a9c481d7616\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.925400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925353 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.925400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925364 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4e4f8a998fcff8f056646c54168639b9-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal\" (UID: \"4e4f8a998fcff8f056646c54168639b9\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:23.925400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:23.925392 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/d7b965d585d2a6f0e3ab4a9c481d7616-config\") pod \"kube-apiserver-proxy-ip-10-0-128-121.ec2.internal\" (UID: \"d7b965d585d2a6f0e3ab4a9c481d7616\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:24.006749 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.006684 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.031184 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.031140 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:24.035790 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.035765 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:24.107183 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.107140 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.207763 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.207730 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.308335 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.308223 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.394403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.394367 2544 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:24.408800 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.408767 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.447337 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.447283 2544 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 12 17:11:24.448010 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.447492 2544 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 12 17:11:24.448010 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.447504 2544 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 12 17:11:24.448010 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.447504 2544 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 12 17:11:24.509824 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.509784 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.523331 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.523256 2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-11 17:06:23 +0000 UTC" deadline="2027-12-20 06:19:35.827486718 +0000 UTC" Jun 12 17:11:24.523331 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.523327 2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13333h8m11.304167242s" Jun 12 17:11:24.610409 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.610369 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.630914 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:24.630879 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e4f8a998fcff8f056646c54168639b9.slice/crio-1eb74cfc0a7c8e92b4a92322c1216b41ce19e380ebe51ac37b8b3a311caa6efa WatchSource:0}: Error finding container 1eb74cfc0a7c8e92b4a92322c1216b41ce19e380ebe51ac37b8b3a311caa6efa: Status 404 returned error can't find the container with id 1eb74cfc0a7c8e92b4a92322c1216b41ce19e380ebe51ac37b8b3a311caa6efa Jun 12 17:11:24.631199 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:24.631176 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7b965d585d2a6f0e3ab4a9c481d7616.slice/crio-9d5a3eef0435e58e6245ea92c9954888dfc73294deb9d2f52f54ddae4b5d10fc WatchSource:0}: Error finding container 9d5a3eef0435e58e6245ea92c9954888dfc73294deb9d2f52f54ddae4b5d10fc: Status 404 returned error can't find the container with id 9d5a3eef0435e58e6245ea92c9954888dfc73294deb9d2f52f54ddae4b5d10fc Jun 12 17:11:24.637071 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.637053 2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 17:11:24.711326 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.711260 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.811815 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.811774 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.912435 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:24.912349 2544 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-121.ec2.internal\" not found" Jun 12 17:11:24.935058 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:24.935034 2544 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:25.023364 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.023336 2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" Jun 12 17:11:25.039923 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.039883 2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 12 17:11:25.040984 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.040953 2544 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" Jun 12 17:11:25.050226 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.050201 2544 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 12 17:11:25.503646 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.503613 2544 apiserver.go:52] "Watching apiserver" Jun 12 17:11:25.519913 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.519595 2544 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 12 17:11:25.521893 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.521860 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-dzhv9","openshift-network-diagnostics/network-check-target-bcj25","openshift-network-operator/iptables-alerter-7cfln","openshift-ovn-kubernetes/ovnkube-node-69swk","kube-system/global-pull-secret-syncer-p25tr","kube-system/konnectivity-agent-hp4d4","kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs","openshift-image-registry/node-ca-6hgck","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal","openshift-multus/multus-lfd6c","openshift-multus/network-metrics-daemon-ws7n9","openshift-cluster-node-tuning-operator/tuned-xg68s","openshift-dns/node-resolver-p25h9"] Jun 12 17:11:25.523258 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.523235 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.524196 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.523689 2544 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 12 17:11:25.524953 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.524936 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:25.525128 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.525095 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:25.525389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.525371 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.525389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.525382 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wfhtm\"" Jun 12 17:11:25.525529 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.525514 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.526333 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.526315 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.527428 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.527406 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.528138 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.528116 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.528233 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.528140 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.528233 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.528182 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 12 17:11:25.528369 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.528238 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5n85w\"" Jun 12 17:11:25.528767 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.528750 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.528850 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.528818 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:25.529699 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.529679 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.529788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.529688 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 12 17:11:25.529954 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.529939 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 12 17:11:25.530182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.530161 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.530470 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.530449 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 12 17:11:25.530681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.530664 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7dhzq\"" Jun 12 17:11:25.530741 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.530705 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.530934 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.530919 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 12 17:11:25.531799 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.531781 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.531927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.531912 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 12 17:11:25.533021 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.533001 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.534416 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.534397 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.535529 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535505 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-conf\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535541 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-systemd\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535567 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-sys\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535586 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-host\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535603 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-kubernetes\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535615 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-run\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535628 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-tuned\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535644 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysconfig\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.535665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535660 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535673 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-var-lib-kubelet\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535688 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjgkt\" (UniqueName: \"kubernetes.io/projected/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-kube-api-access-qjgkt\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535706 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535722 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-modprobe-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535722 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535737 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-lib-modules\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.536231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.535751 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-tmp\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.537007 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.536987 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 12 17:11:25.537097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537087 2544 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:25.537158 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537094 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 12 17:11:25.537211 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537155 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.537211 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537087 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-6njqj\"" Jun 12 17:11:25.537399 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537380 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 12 17:11:25.537483 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537402 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 12 17:11:25.537483 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537451 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-42dmr\"" Jun 12 17:11:25.537483 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537470 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 12 17:11:25.537618 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537577 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.538118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.537972 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.538118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538034 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.538586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538413 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 12 17:11:25.538586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538426 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cvdfl\"" Jun 12 17:11:25.538586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538499 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.538782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538638 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.538782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538720 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.538782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.538763 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-2plmn\"" Jun 12 17:11:25.539701 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.539682 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dfn4s\"" Jun 12 17:11:25.539804 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.539765 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 12 17:11:25.540043 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.540011 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.540138 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.540081 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:25.540231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.540211 2544 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 12 17:11:25.540577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.540504 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 12 17:11:25.540577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.540554 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-bfmlj\"" Jun 12 17:11:25.541157 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.541142 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 12 17:11:25.562754 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.562727 2544 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hd4pb" Jun 12 17:11:25.571881 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.571844 2544 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hd4pb" Jun 12 17:11:25.602807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.602749 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" event={"ID":"d7b965d585d2a6f0e3ab4a9c481d7616","Type":"ContainerStarted","Data":"9d5a3eef0435e58e6245ea92c9954888dfc73294deb9d2f52f54ddae4b5d10fc"} Jun 12 17:11:25.603822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.603799 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" event={"ID":"4e4f8a998fcff8f056646c54168639b9","Type":"ContainerStarted","Data":"1eb74cfc0a7c8e92b4a92322c1216b41ce19e380ebe51ac37b8b3a311caa6efa"} Jun 12 17:11:25.624603 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.624570 2544 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jun 12 17:11:25.636100 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636062 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-systemd-units\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.636272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636110 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2acee30e-f461-4ee5-9a33-03ee6e876d3e-agent-certs\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.636272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636175 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-system-cni-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.636272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636212 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-cnibin\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.636272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636237 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2bwk\" (UniqueName: \"kubernetes.io/projected/e159f176-bd80-4a5a-93c2-abcbd609ec4f-kube-api-access-t2bwk\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.636272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636260 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2acee30e-f461-4ee5-9a33-03ee6e876d3e-konnectivity-ca\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636282 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-sys\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636338 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-host\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636353 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636370 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-bin\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636394 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-netd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636414 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/028bb6d9-6629-443c-a31d-83abf2688151-host\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636436 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-kubelet\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636455 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-host\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636505 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-sys\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636540 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-socket-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636558 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-device-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636579 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/028bb6d9-6629-443c-a31d-83abf2688151-serviceca\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636606 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysconfig\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636632 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636670 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636692 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysconfig\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636702 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636727 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-hostroot\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636750 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56xc9\" (UniqueName: \"kubernetes.io/projected/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-kube-api-access-56xc9\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636787 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-tmp\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636813 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cnibin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636834 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-os-release\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.636872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636859 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-bin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636882 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636907 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-ovn\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.636981 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-node-log\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637005 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-netns\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637028 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637055 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-conf-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637071 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-run\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637086 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-tuned\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637107 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-kubelet\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637167 2544 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637249 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-run\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637181 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9c341d1-e1ea-4164-8549-8824c5b6ae60-host-slash\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637339 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-var-lib-kubelet\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637359 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjgkt\" (UniqueName: \"kubernetes.io/projected/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-kube-api-access-qjgkt\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.637385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637382 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cni-binary-copy\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637406 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-socket-dir-parent\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637431 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-k8s-cni-cncf-io\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637455 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-registration-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637480 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ea666b9-869f-479d-a9de-d27c52931609-ovn-node-metrics-cert\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637532 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-var-lib-kubelet\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637520 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zkhx\" (UniqueName: \"kubernetes.io/projected/1ea666b9-869f-479d-a9de-d27c52931609-kube-api-access-4zkhx\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637585 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637618 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e159f176-bd80-4a5a-93c2-abcbd609ec4f-hosts-file\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637695 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-conf\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637728 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-var-lib-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637756 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwtlx\" (UniqueName: \"kubernetes.io/projected/028bb6d9-6629-443c-a31d-83abf2688151-kube-api-access-lwtlx\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637781 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-multus\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637813 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-etc-kubernetes\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637868 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-conf\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637870 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637907 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-systemd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637927 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637949 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-env-overrides\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637972 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-os-release\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637986 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-sysctl-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.637993 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-system-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638021 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-netns\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638061 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638093 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638136 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-modprobe-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638194 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-lib-modules\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638230 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-modprobe-d\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638268 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-kubelet-config\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638273 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-lib-modules\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638326 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-sys-fs\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638397 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8bjg\" (UniqueName: \"kubernetes.io/projected/b9c341d1-e1ea-4164-8549-8824c5b6ae60-kube-api-access-h8bjg\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638421 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-etc-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.638812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638447 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-systemd\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638470 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-slash\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638492 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-log-socket\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638514 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-config\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638537 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-script-lib\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638540 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-systemd\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638574 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-dbus\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638656 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638690 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-kubernetes\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638717 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hk4hc\" (UniqueName: \"kubernetes.io/projected/3aba6f38-a65a-43a8-86db-43d96ad371ee-kube-api-access-hk4hc\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638744 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e159f176-bd80-4a5a-93c2-abcbd609ec4f-tmp-dir\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638768 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9c341d1-e1ea-4164-8549-8824c5b6ae60-iptables-alerter-script\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638792 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638815 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-daemon-config\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638838 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-multus-certs\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638863 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.639633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638887 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm9tb\" (UniqueName: \"kubernetes.io/projected/204d8c01-20a3-4b41-993d-bf31f7c55a44-kube-api-access-bm9tb\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.640434 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.638914 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qscss\" (UniqueName: \"kubernetes.io/projected/fadb0b7d-f055-467b-a67a-f034a8cfee0a-kube-api-access-qscss\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.640434 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.639051 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-kubernetes\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.640532 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.640474 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-etc-tuned\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.640621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.640600 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-tmp\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.643383 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.643359 2544 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:25.644853 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.644813 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:25.644986 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.644859 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:25.644986 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.644874 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:25.644986 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.644956 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:26.144934885 +0000 UTC m=+3.017582991 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:25.647474 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.647446 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjgkt\" (UniqueName: \"kubernetes.io/projected/a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13-kube-api-access-qjgkt\") pod \"tuned-xg68s\" (UID: \"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13\") " pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.739461 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739423 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-os-release\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.739461 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739456 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-system-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739476 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-netns\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739490 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739519 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-kubelet-config\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739545 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-sys-fs\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739555 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-netns\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739557 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-os-release\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739606 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-kubelet-config\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739564 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-system-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739567 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8bjg\" (UniqueName: \"kubernetes.io/projected/b9c341d1-e1ea-4164-8549-8824c5b6ae60-kube-api-access-h8bjg\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739638 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-sys-fs\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.739657 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739662 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-etc-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.739700 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739690 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-slash\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.739724 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:26.239704626 +0000 UTC m=+3.112352736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739735 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-slash\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739743 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-log-socket\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739762 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-etc-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739773 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-config\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739805 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-script-lib\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739827 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-dbus\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739834 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-log-socket\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739843 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739892 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hk4hc\" (UniqueName: \"kubernetes.io/projected/3aba6f38-a65a-43a8-86db-43d96ad371ee-kube-api-access-hk4hc\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.739899 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739922 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e159f176-bd80-4a5a-93c2-abcbd609ec4f-tmp-dir\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:25.739952 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:26.239938635 +0000 UTC m=+3.112586722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.739969 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9c341d1-e1ea-4164-8549-8824c5b6ae60-iptables-alerter-script\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740001 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.740385 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740022 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-daemon-config\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740044 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-multus-certs\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740069 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740094 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm9tb\" (UniqueName: \"kubernetes.io/projected/204d8c01-20a3-4b41-993d-bf31f7c55a44-kube-api-access-bm9tb\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740119 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qscss\" (UniqueName: \"kubernetes.io/projected/fadb0b7d-f055-467b-a67a-f034a8cfee0a-kube-api-access-qscss\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740142 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-systemd-units\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740166 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2acee30e-f461-4ee5-9a33-03ee6e876d3e-agent-certs\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740188 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-system-cni-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740214 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-cnibin\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740236 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-cni-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740237 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2bwk\" (UniqueName: \"kubernetes.io/projected/e159f176-bd80-4a5a-93c2-abcbd609ec4f-kube-api-access-t2bwk\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740266 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2acee30e-f461-4ee5-9a33-03ee6e876d3e-konnectivity-ca\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740285 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740330 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-bin\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740354 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-netd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740373 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/028bb6d9-6629-443c-a31d-83abf2688151-host\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740392 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-kubelet\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.741429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740412 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-socket-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740455 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-device-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740461 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/40d265f2-894f-43a3-8643-36b1157cf081-dbus\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740479 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-config\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740489 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-system-cni-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740189 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e159f176-bd80-4a5a-93c2-abcbd609ec4f-tmp-dir\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740507 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-bin\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740527 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-ovnkube-script-lib\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740482 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/028bb6d9-6629-443c-a31d-83abf2688151-serviceca\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740560 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9c341d1-e1ea-4164-8549-8824c5b6ae60-iptables-alerter-script\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740571 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740636 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740741 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-multus-certs\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740778 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740804 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740829 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-hostroot\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740829 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/028bb6d9-6629-443c-a31d-83abf2688151-host\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.742257 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740854 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56xc9\" (UniqueName: \"kubernetes.io/projected/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-kube-api-access-56xc9\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740871 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740883 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cnibin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740893 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/028bb6d9-6629-443c-a31d-83abf2688151-serviceca\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740906 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-cnibin\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740947 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-cni-netd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740948 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-kubelet\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740979 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-systemd-units\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.740995 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-device-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741013 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-os-release\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741031 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-daemon-config\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741041 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-bin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741034 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-kubelet-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741067 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741070 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/2acee30e-f461-4ee5-9a33-03ee6e876d3e-konnectivity-ca\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741074 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-etc-selinux\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741115 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-ovn\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741124 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-bin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741127 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-socket-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741140 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-node-log\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741146 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-os-release\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741165 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-netns\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741177 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-ovn\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741197 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cnibin\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741215 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-node-log\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741229 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-hostroot\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741245 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741251 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741275 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-run-netns\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741338 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-conf-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741371 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-conf-dir\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741398 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-kubelet\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741426 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9c341d1-e1ea-4164-8549-8824c5b6ae60-host-slash\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741437 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-kubelet\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741443 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3aba6f38-a65a-43a8-86db-43d96ad371ee-tuning-conf-dir\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741464 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9c341d1-e1ea-4164-8549-8824c5b6ae60-host-slash\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.743771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741481 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cni-binary-copy\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741503 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-socket-dir-parent\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741513 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741539 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-multus-socket-dir-parent\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741547 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-k8s-cni-cncf-io\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741568 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-registration-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741585 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ea666b9-869f-479d-a9de-d27c52931609-ovn-node-metrics-cert\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741598 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/204d8c01-20a3-4b41-993d-bf31f7c55a44-registration-dir\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741604 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zkhx\" (UniqueName: \"kubernetes.io/projected/1ea666b9-869f-479d-a9de-d27c52931609-kube-api-access-4zkhx\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741629 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741653 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e159f176-bd80-4a5a-93c2-abcbd609ec4f-hosts-file\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741678 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-var-lib-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741568 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-run-k8s-cni-cncf-io\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741720 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwtlx\" (UniqueName: \"kubernetes.io/projected/028bb6d9-6629-443c-a31d-83abf2688151-kube-api-access-lwtlx\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741749 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-multus\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741764 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/e159f176-bd80-4a5a-93c2-abcbd609ec4f-hosts-file\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741773 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-etc-kubernetes\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741724 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-var-lib-openvswitch\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741799 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-systemd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741825 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741842 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-etc-kubernetes\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741848 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-env-overrides\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741876 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/fadb0b7d-f055-467b-a67a-f034a8cfee0a-cni-binary-copy\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741880 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/fadb0b7d-f055-467b-a67a-f034a8cfee0a-host-var-lib-cni-multus\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.741920 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.742012 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/1ea666b9-869f-479d-a9de-d27c52931609-run-systemd\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.742086 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3aba6f38-a65a-43a8-86db-43d96ad371ee-cni-binary-copy\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.742173 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/1ea666b9-869f-479d-a9de-d27c52931609-env-overrides\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.743640 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/2acee30e-f461-4ee5-9a33-03ee6e876d3e-agent-certs\") pod \"konnectivity-agent-hp4d4\" (UID: \"2acee30e-f461-4ee5-9a33-03ee6e876d3e\") " pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.744970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.744062 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/1ea666b9-869f-479d-a9de-d27c52931609-ovn-node-metrics-cert\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.755511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.755400 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8bjg\" (UniqueName: \"kubernetes.io/projected/b9c341d1-e1ea-4164-8549-8824c5b6ae60-kube-api-access-h8bjg\") pod \"iptables-alerter-7cfln\" (UID: \"b9c341d1-e1ea-4164-8549-8824c5b6ae60\") " pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.755665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.755559 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qscss\" (UniqueName: \"kubernetes.io/projected/fadb0b7d-f055-467b-a67a-f034a8cfee0a-kube-api-access-qscss\") pod \"multus-lfd6c\" (UID: \"fadb0b7d-f055-467b-a67a-f034a8cfee0a\") " pod="openshift-multus/multus-lfd6c" Jun 12 17:11:25.755665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.755625 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwtlx\" (UniqueName: \"kubernetes.io/projected/028bb6d9-6629-443c-a31d-83abf2688151-kube-api-access-lwtlx\") pod \"node-ca-6hgck\" (UID: \"028bb6d9-6629-443c-a31d-83abf2688151\") " pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.755951 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.755926 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2bwk\" (UniqueName: \"kubernetes.io/projected/e159f176-bd80-4a5a-93c2-abcbd609ec4f-kube-api-access-t2bwk\") pod \"node-resolver-p25h9\" (UID: \"e159f176-bd80-4a5a-93c2-abcbd609ec4f\") " pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.756894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.756869 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hk4hc\" (UniqueName: \"kubernetes.io/projected/3aba6f38-a65a-43a8-86db-43d96ad371ee-kube-api-access-hk4hc\") pod \"multus-additional-cni-plugins-dzhv9\" (UID: \"3aba6f38-a65a-43a8-86db-43d96ad371ee\") " pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.757039 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.756870 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm9tb\" (UniqueName: \"kubernetes.io/projected/204d8c01-20a3-4b41-993d-bf31f7c55a44-kube-api-access-bm9tb\") pod \"aws-ebs-csi-driver-node-c2wxs\" (UID: \"204d8c01-20a3-4b41-993d-bf31f7c55a44\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.757561 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.757529 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zkhx\" (UniqueName: \"kubernetes.io/projected/1ea666b9-869f-479d-a9de-d27c52931609-kube-api-access-4zkhx\") pod \"ovnkube-node-69swk\" (UID: \"1ea666b9-869f-479d-a9de-d27c52931609\") " pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.758449 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.758431 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56xc9\" (UniqueName: \"kubernetes.io/projected/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-kube-api-access-56xc9\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:25.835613 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.835580 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xg68s" Jun 12 17:11:25.845535 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.845500 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7cfln" Jun 12 17:11:25.855549 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.855511 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:25.861522 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.861493 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:25.870388 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.870350 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" Jun 12 17:11:25.876635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.876424 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" Jun 12 17:11:25.882263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.882240 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6hgck" Jun 12 17:11:25.887962 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.887939 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p25h9" Jun 12 17:11:25.892745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:25.892719 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-lfd6c" Jun 12 17:11:26.145025 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.144982 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:26.145211 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.145166 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:26.145211 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.145189 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:26.145338 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.145202 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:26.145338 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.145281 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:27.145266999 +0000 UTC m=+4.017915086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:26.245910 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.245875 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:26.246108 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.245935 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:26.246108 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.246053 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:26.246218 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.246141 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:27.246118619 +0000 UTC m=+4.118766725 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:26.246218 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.246058 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:26.246218 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:26.246211 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:27.246194223 +0000 UTC m=+4.118842325 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:26.573882 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.573840 2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-11 17:06:25 +0000 UTC" deadline="2028-02-03 12:21:39.574939886 +0000 UTC" Jun 12 17:11:26.573882 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.573876 2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14419h10m13.001067153s" Jun 12 17:11:26.591677 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.591453 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ea666b9_869f_479d_a9de_d27c52931609.slice/crio-9091b55a00a1727f2528037108d689d479c906187c2fc682127ec171d8b97ad6 WatchSource:0}: Error finding container 9091b55a00a1727f2528037108d689d479c906187c2fc682127ec171d8b97ad6: Status 404 returned error can't find the container with id 9091b55a00a1727f2528037108d689d479c906187c2fc682127ec171d8b97ad6 Jun 12 17:11:26.592805 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.592779 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2acee30e_f461_4ee5_9a33_03ee6e876d3e.slice/crio-9bd7d781f11ddf817d9fd9c8af59f77d2e0d5803a4b86857f1433bb035124936 WatchSource:0}: Error finding container 9bd7d781f11ddf817d9fd9c8af59f77d2e0d5803a4b86857f1433bb035124936: Status 404 returned error can't find the container with id 9bd7d781f11ddf817d9fd9c8af59f77d2e0d5803a4b86857f1433bb035124936 Jun 12 17:11:26.594665 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.594626 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9c341d1_e1ea_4164_8549_8824c5b6ae60.slice/crio-ff6ae91328dc29b7fa11279fee1eaeb216cec5ec7d9694e46149df391d8c7fb8 WatchSource:0}: Error finding container ff6ae91328dc29b7fa11279fee1eaeb216cec5ec7d9694e46149df391d8c7fb8: Status 404 returned error can't find the container with id ff6ae91328dc29b7fa11279fee1eaeb216cec5ec7d9694e46149df391d8c7fb8 Jun 12 17:11:26.599071 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.599044 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8f3ac3d_a3de_4cc9_b4bc_f16f3afb1f13.slice/crio-795cb60ea4a794659d942be41c81419192c5bac8ed9766cca38f7b8fabe7b8c7 WatchSource:0}: Error finding container 795cb60ea4a794659d942be41c81419192c5bac8ed9766cca38f7b8fabe7b8c7: Status 404 returned error can't find the container with id 795cb60ea4a794659d942be41c81419192c5bac8ed9766cca38f7b8fabe7b8c7 Jun 12 17:11:26.599735 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.599709 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod028bb6d9_6629_443c_a31d_83abf2688151.slice/crio-1f1fc2f4c11cfea5113c299f82f29acedd44199cd25e06b94a1aeada615899ed WatchSource:0}: Error finding container 1f1fc2f4c11cfea5113c299f82f29acedd44199cd25e06b94a1aeada615899ed: Status 404 returned error can't find the container with id 1f1fc2f4c11cfea5113c299f82f29acedd44199cd25e06b94a1aeada615899ed Jun 12 17:11:26.600803 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.600770 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode159f176_bd80_4a5a_93c2_abcbd609ec4f.slice/crio-1201bdf5c4cccabe3b1b3749b5ecc71fdf0be130b756c8cab155e6a894c03aa7 WatchSource:0}: Error finding container 1201bdf5c4cccabe3b1b3749b5ecc71fdf0be130b756c8cab155e6a894c03aa7: Status 404 returned error can't find the container with id 1201bdf5c4cccabe3b1b3749b5ecc71fdf0be130b756c8cab155e6a894c03aa7 Jun 12 17:11:26.601632 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.601599 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod204d8c01_20a3_4b41_993d_bf31f7c55a44.slice/crio-66d5a12ba56fd0bc34693e4fe7f96cc3001d128b0c0c3f8999d696df45ed87e4 WatchSource:0}: Error finding container 66d5a12ba56fd0bc34693e4fe7f96cc3001d128b0c0c3f8999d696df45ed87e4: Status 404 returned error can't find the container with id 66d5a12ba56fd0bc34693e4fe7f96cc3001d128b0c0c3f8999d696df45ed87e4 Jun 12 17:11:26.603506 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.603461 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfadb0b7d_f055_467b_a67a_f034a8cfee0a.slice/crio-5c16013ccedbc0ce7f5807402a46a491ca0d112787d71cc9b3388e54e7b6dd99 WatchSource:0}: Error finding container 5c16013ccedbc0ce7f5807402a46a491ca0d112787d71cc9b3388e54e7b6dd99: Status 404 returned error can't find the container with id 5c16013ccedbc0ce7f5807402a46a491ca0d112787d71cc9b3388e54e7b6dd99 Jun 12 17:11:26.605276 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:26.605153 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aba6f38_a65a_43a8_86db_43d96ad371ee.slice/crio-735117d6684495f292d15e3d60d8641782961317d650da9eb64c1b44a4dd596e WatchSource:0}: Error finding container 735117d6684495f292d15e3d60d8641782961317d650da9eb64c1b44a4dd596e: Status 404 returned error can't find the container with id 735117d6684495f292d15e3d60d8641782961317d650da9eb64c1b44a4dd596e Jun 12 17:11:26.607868 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.607835 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hgck" event={"ID":"028bb6d9-6629-443c-a31d-83abf2688151","Type":"ContainerStarted","Data":"1f1fc2f4c11cfea5113c299f82f29acedd44199cd25e06b94a1aeada615899ed"} Jun 12 17:11:26.609633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.609542 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xg68s" event={"ID":"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13","Type":"ContainerStarted","Data":"795cb60ea4a794659d942be41c81419192c5bac8ed9766cca38f7b8fabe7b8c7"} Jun 12 17:11:26.611023 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.610998 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7cfln" event={"ID":"b9c341d1-e1ea-4164-8549-8824c5b6ae60","Type":"ContainerStarted","Data":"ff6ae91328dc29b7fa11279fee1eaeb216cec5ec7d9694e46149df391d8c7fb8"} Jun 12 17:11:26.612333 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.612283 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hp4d4" event={"ID":"2acee30e-f461-4ee5-9a33-03ee6e876d3e","Type":"ContainerStarted","Data":"9bd7d781f11ddf817d9fd9c8af59f77d2e0d5803a4b86857f1433bb035124936"} Jun 12 17:11:26.614038 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.614012 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"9091b55a00a1727f2528037108d689d479c906187c2fc682127ec171d8b97ad6"} Jun 12 17:11:26.615195 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.615175 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lfd6c" event={"ID":"fadb0b7d-f055-467b-a67a-f034a8cfee0a","Type":"ContainerStarted","Data":"5c16013ccedbc0ce7f5807402a46a491ca0d112787d71cc9b3388e54e7b6dd99"} Jun 12 17:11:26.616418 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.616397 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" event={"ID":"204d8c01-20a3-4b41-993d-bf31f7c55a44","Type":"ContainerStarted","Data":"66d5a12ba56fd0bc34693e4fe7f96cc3001d128b0c0c3f8999d696df45ed87e4"} Jun 12 17:11:26.618381 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.618362 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p25h9" event={"ID":"e159f176-bd80-4a5a-93c2-abcbd609ec4f","Type":"ContainerStarted","Data":"1201bdf5c4cccabe3b1b3749b5ecc71fdf0be130b756c8cab155e6a894c03aa7"} Jun 12 17:11:26.990283 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:26.990247 2544 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 12 17:11:27.154954 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.154373 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:27.154954 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.154545 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:27.154954 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.154570 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:27.154954 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.154584 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:27.154954 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.154646 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:29.154626473 +0000 UTC m=+6.027274563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:27.255184 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.255146 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:27.255375 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.255215 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:27.255435 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.255404 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:27.255485 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.255472 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:29.255452899 +0000 UTC m=+6.128100989 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:27.255910 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.255797 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:27.255910 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.255878 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:29.255860171 +0000 UTC m=+6.128508280 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:27.574630 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.574478 2544 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-11 17:06:25 +0000 UTC" deadline="2027-12-17 13:18:42.324479881 +0000 UTC" Jun 12 17:11:27.574630 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.574541 2544 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13268h7m14.749943024s" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.598417 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.598556 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.598562 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.598658 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.598424 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:27.598927 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:27.598744 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:27.626091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.626049 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerStarted","Data":"735117d6684495f292d15e3d60d8641782961317d650da9eb64c1b44a4dd596e"} Jun 12 17:11:27.639341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.639284 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" event={"ID":"d7b965d585d2a6f0e3ab4a9c481d7616","Type":"ContainerStarted","Data":"39672be07dba632548c0c9bdc20841067f760a7352e5a63196c9165a6e6fb4c9"} Jun 12 17:11:27.660918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.658424 2544 generic.go:356] "Generic (PLEG): container finished" podID="4e4f8a998fcff8f056646c54168639b9" containerID="a950d9451188789a1fe07e629ef20d1daf9d578f413c5efce6bfdf2e6a307683" exitCode=0 Jun 12 17:11:27.660918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.658493 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" event={"ID":"4e4f8a998fcff8f056646c54168639b9","Type":"ContainerDied","Data":"a950d9451188789a1fe07e629ef20d1daf9d578f413c5efce6bfdf2e6a307683"} Jun 12 17:11:27.660918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:27.660470 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-121.ec2.internal" podStartSLOduration=2.660449741 podStartE2EDuration="2.660449741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:27.659455297 +0000 UTC m=+4.532103406" watchObservedRunningTime="2026-06-12 17:11:27.660449741 +0000 UTC m=+4.533097852" Jun 12 17:11:28.688895 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:28.688854 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" event={"ID":"4e4f8a998fcff8f056646c54168639b9","Type":"ContainerStarted","Data":"8261ccbea449dd6859b3a3b4fb417fdd20cc941b737deb9187c5a424d5771f77"} Jun 12 17:11:29.176181 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.175528 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:29.176181 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.175696 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:29.176181 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.175718 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:29.176181 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.175731 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:29.176181 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.175786 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:33.175768098 +0000 UTC m=+10.048416207 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:29.276284 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.276246 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:29.276484 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.276331 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:29.276484 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.276459 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:29.276592 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.276521 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:33.276502964 +0000 UTC m=+10.149151065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:29.276888 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.276871 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:29.276953 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.276920 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:33.276905046 +0000 UTC m=+10.149553137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:29.597505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.597422 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:29.597671 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.597425 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:29.597671 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.597565 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:29.597671 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:29.597579 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:29.597671 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.597660 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:29.597878 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:29.597750 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:31.597109 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:31.597050 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:31.597738 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:31.597050 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:31.597738 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:31.597216 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:31.597738 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:31.597261 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:31.597738 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:31.597063 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:31.597738 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:31.597385 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:33.208775 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.208735 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:33.209267 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.208930 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:33.209267 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.208960 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:33.209267 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.208974 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:33.209267 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.209033 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:41.209011248 +0000 UTC m=+18.081659338 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:33.309472 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.309437 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:33.309643 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.309498 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:33.309643 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.309602 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:33.309771 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.309657 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:41.309643141 +0000 UTC m=+18.182291228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:33.310049 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.310028 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:33.310143 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.310081 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:41.310065524 +0000 UTC m=+18.182713611 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:33.597550 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.597474 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:33.597718 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.597593 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:33.597718 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.597671 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:33.597826 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.597801 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:33.597897 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:33.597846 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:33.597957 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:33.597906 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:35.597915 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:35.597775 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:35.597915 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:35.597909 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:35.598465 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:35.598322 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:35.598465 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:35.598438 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:35.598573 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:35.598477 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:35.598573 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:35.598547 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:37.597689 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:37.596936 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:37.597689 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:37.597076 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:37.597689 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:37.597495 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:37.597689 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:37.597629 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:37.598342 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:37.597793 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:37.598342 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:37.597882 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:39.597094 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:39.597054 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:39.597574 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:39.597185 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:39.597574 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:39.597185 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:39.597574 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:39.597064 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:39.597574 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:39.597312 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:39.597574 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:39.597399 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:41.274198 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.274159 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:41.274753 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.274308 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:41.274753 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.274330 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:41.274753 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.274344 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:41.274753 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.274407 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:11:57.274388057 +0000 UTC m=+34.147036180 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:41.375321 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.375273 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:41.375508 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.375343 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:41.375508 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.375444 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:41.375508 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.375443 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:41.375625 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.375516 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:57.375494885 +0000 UTC m=+34.248142981 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:41.375625 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.375541 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:57.375529973 +0000 UTC m=+34.248178061 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:41.596951 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.596862 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:41.597109 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.596862 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:41.597109 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.597013 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:41.597211 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.597100 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:41.597211 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:41.596862 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:41.597211 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:41.597192 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:43.598189 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:43.598143 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:43.598665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:43.598217 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:43.598665 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:43.598310 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:43.598665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:43.598545 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:43.598665 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:43.598638 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:43.598820 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:43.598706 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:45.597251 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.597089 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:45.598013 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.597095 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:45.598013 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.597095 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:45.598013 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:45.597381 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:45.598013 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:45.597547 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:45.598013 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:45.597664 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:45.727141 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.727106 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-lfd6c" event={"ID":"fadb0b7d-f055-467b-a67a-f034a8cfee0a","Type":"ContainerStarted","Data":"3ca7b4b2afab927afab9bf85085192b3c39f62563c72f054f76d08054b392846"} Jun 12 17:11:45.728944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.728907 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" event={"ID":"204d8c01-20a3-4b41-993d-bf31f7c55a44","Type":"ContainerStarted","Data":"a03d4de10a594264f68b5c66c4c4f7f16f85739793315510767ae2ead3562c4d"} Jun 12 17:11:45.730661 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.730633 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p25h9" event={"ID":"e159f176-bd80-4a5a-93c2-abcbd609ec4f","Type":"ContainerStarted","Data":"aae1b6fe8dd30d9774dfa2896bda14b2a59d0d127bae1fbd025e67dd50586fd3"} Jun 12 17:11:45.732607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.732577 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="17f1de5e8cdf674ff817036b0271aaa558622f2a9f751b341c3d9a011d5b188b" exitCode=0 Jun 12 17:11:45.732727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.732684 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"17f1de5e8cdf674ff817036b0271aaa558622f2a9f751b341c3d9a011d5b188b"} Jun 12 17:11:45.734390 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.734362 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6hgck" event={"ID":"028bb6d9-6629-443c-a31d-83abf2688151","Type":"ContainerStarted","Data":"bc19bf5e7f801fde5c80c324a11772c50657c1e06cab9c43f0ce24143380a7e7"} Jun 12 17:11:45.736213 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.736191 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xg68s" event={"ID":"a8f3ac3d-a3de-4cc9-b4bc-f16f3afb1f13","Type":"ContainerStarted","Data":"3d89e7f27343b359423f75506c33c41a50ef199ae221e45366f20990173e0a87"} Jun 12 17:11:45.737728 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.737702 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-hp4d4" event={"ID":"2acee30e-f461-4ee5-9a33-03ee6e876d3e","Type":"ContainerStarted","Data":"fe602f0724826259d7492e0cb9521583b06d32fb84894e2e67c27b6398db7094"} Jun 12 17:11:45.739496 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.739473 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:11:45.739929 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.739909 2544 generic.go:356] "Generic (PLEG): container finished" podID="1ea666b9-869f-479d-a9de-d27c52931609" containerID="f9b0a22eaa7253a472220f0def2d1265161608720b1af3a3b11ee94a04a6c754" exitCode=1 Jun 12 17:11:45.740025 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.739942 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerDied","Data":"f9b0a22eaa7253a472220f0def2d1265161608720b1af3a3b11ee94a04a6c754"} Jun 12 17:11:45.740085 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.740021 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"edabafd3d2d25e45c0a6753e7d3e61a165f2f7b4a7f232a8382a140f61f44907"} Jun 12 17:11:45.753951 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.753903 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-121.ec2.internal" podStartSLOduration=20.753883101 podStartE2EDuration="20.753883101s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:28.707743726 +0000 UTC m=+5.580391833" watchObservedRunningTime="2026-06-12 17:11:45.753883101 +0000 UTC m=+22.626531210" Jun 12 17:11:45.785104 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.785053 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-lfd6c" podStartSLOduration=4.097635501 podStartE2EDuration="22.785037682s" totalImagesPullingTime="18.687402181s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:45.754048899 +0000 UTC m=+22.626697000" watchObservedRunningTime="2026-06-12 17:11:45.785037682 +0000 UTC m=+22.657685790" Jun 12 17:11:45.820719 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.818821 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-p25h9" podStartSLOduration=4.189218789 podStartE2EDuration="22.818801501s" totalImagesPullingTime="18.629582712s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:45.818582869 +0000 UTC m=+22.691231004" watchObservedRunningTime="2026-06-12 17:11:45.818801501 +0000 UTC m=+22.691449609" Jun 12 17:11:45.820719 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.819616 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6hgck" podStartSLOduration=4.188868544 podStartE2EDuration="22.819598852s" totalImagesPullingTime="18.630730308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:45.802407012 +0000 UTC m=+22.675055120" watchObservedRunningTime="2026-06-12 17:11:45.819598852 +0000 UTC m=+22.692246963" Jun 12 17:11:45.838824 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:45.838776 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xg68s" podStartSLOduration=4.191812954 podStartE2EDuration="22.838755493s" totalImagesPullingTime="18.646942539s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:45.838320596 +0000 UTC m=+22.710968696" watchObservedRunningTime="2026-06-12 17:11:45.838755493 +0000 UTC m=+22.711403606" Jun 12 17:11:46.092119 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.092004 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:46.092660 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.092639 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:46.114795 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.114743 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-hp4d4" podStartSLOduration=9.000113017 podStartE2EDuration="23.114725373s" totalImagesPullingTime="14.114612356s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:45.853520992 +0000 UTC m=+22.726169101" watchObservedRunningTime="2026-06-12 17:11:46.114725373 +0000 UTC m=+22.987373484" Jun 12 17:11:46.513392 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.513192 2544 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 12 17:11:46.594016 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.593902 2544 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-12T17:11:46.513386918Z","UUID":"889ea124-cc90-463a-9575-d77b9c24152b","Handler":null,"Name":"","Endpoint":""} Jun 12 17:11:46.595659 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.595641 2544 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 12 17:11:46.595744 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.595679 2544 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 12 17:11:46.743949 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.743911 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7cfln" event={"ID":"b9c341d1-e1ea-4164-8549-8824c5b6ae60","Type":"ContainerStarted","Data":"1b92d10f81697779bc4130a8c1b2a9d22b7696055139e296795097ec4a18b654"} Jun 12 17:11:46.747371 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.747343 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:11:46.747789 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.747761 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"089f8eb30919df6f8ad3efa245727b1f15093c8020556649c839d2634c0d4782"} Jun 12 17:11:46.747871 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.747852 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"7efa593749413b43934a505b9ad2561d556331e58cdd7497098e5d28c2f3ee66"} Jun 12 17:11:46.747924 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.747906 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"f6459b48d43783bfcc6520e731a9e9c66f7ef059347fe4e16611b06c9dffd19e"} Jun 12 17:11:46.747989 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.747965 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"3a9b4c4a2206bb8012763aca2dcb3617f9264f83da2cd4d7b17a7667de255bc1"} Jun 12 17:11:46.749718 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.749691 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" event={"ID":"204d8c01-20a3-4b41-993d-bf31f7c55a44","Type":"ContainerStarted","Data":"1ed27e18062f7920bc18795e7cc48cdd490243fad67e3a8a5641caf544885ffb"} Jun 12 17:11:46.750343 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.750315 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:46.750870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.750850 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-hp4d4" Jun 12 17:11:46.762415 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:46.762364 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7cfln" podStartSLOduration=5.111646672 podStartE2EDuration="23.762345032s" totalImagesPullingTime="18.65069836s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:46.761607113 +0000 UTC m=+23.634255221" watchObservedRunningTime="2026-06-12 17:11:46.762345032 +0000 UTC m=+23.634993141" Jun 12 17:11:47.597263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:47.597227 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:47.597263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:47.597228 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:47.597529 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:47.597227 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:47.597529 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:47.597367 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:47.597529 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:47.597425 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:47.597529 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:47.597466 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:47.754408 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:47.754313 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" event={"ID":"204d8c01-20a3-4b41-993d-bf31f7c55a44","Type":"ContainerStarted","Data":"c101e354db04e7adfaaca591b51b57beeb2352b50fae348731c517117c2cc5d4"} Jun 12 17:11:47.773760 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:47.773708 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-c2wxs" podStartSLOduration=4.317762833 podStartE2EDuration="24.773693443s" totalImagesPullingTime="20.45593061s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:47.773501217 +0000 UTC m=+24.646149317" watchObservedRunningTime="2026-06-12 17:11:47.773693443 +0000 UTC m=+24.646341552" Jun 12 17:11:48.759082 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:48.759049 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:11:48.759596 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:48.759364 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"cd3f27e7861b1d65dec7888b1d93a77cdc2972b9e93d5967a763139cf028f170"} Jun 12 17:11:49.597133 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:49.597101 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:49.597133 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:49.597118 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:49.597408 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:49.597107 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:49.597408 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:49.597206 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:49.597408 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:49.597264 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:49.597408 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:49.597358 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:50.767158 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.766975 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:11:50.767819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.767549 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"26eb13a9bf58f26371c74c22af785dd3da3bc6e9165e98397b19fcd879ba221b"} Jun 12 17:11:50.767904 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.767846 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:50.767965 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.767914 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:50.768023 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.767964 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:50.768132 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.768112 2544 scope.go:117] "RemoveContainer" containerID="f9b0a22eaa7253a472220f0def2d1265161608720b1af3a3b11ee94a04a6c754" Jun 12 17:11:50.785325 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.785275 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:50.785444 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:50.785430 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:11:51.596834 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.596799 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:51.596834 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.596818 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:51.597099 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.596817 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:51.597099 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:51.596929 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:51.597099 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:51.597057 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:51.597247 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:51.597172 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:51.776758 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.776726 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:11:51.777246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.777174 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" event={"ID":"1ea666b9-869f-479d-a9de-d27c52931609","Type":"ContainerStarted","Data":"77af43312bf12a5c976ff632f40c0bd6c2891804c8274d547078020199641823"} Jun 12 17:11:51.814276 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:51.814210 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" podStartSLOduration=10.06693646 podStartE2EDuration="28.814187148s" totalImagesPullingTime="18.747250688s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:11:51.812467304 +0000 UTC m=+28.685115433" watchObservedRunningTime="2026-06-12 17:11:51.814187148 +0000 UTC m=+28.686835258" Jun 12 17:11:52.305557 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.305512 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p25tr"] Jun 12 17:11:52.305725 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.305651 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:52.305792 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:52.305758 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:52.309393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.309358 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcj25"] Jun 12 17:11:52.309540 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.309487 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:52.309632 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:52.309604 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:52.310158 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.310128 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ws7n9"] Jun 12 17:11:52.310336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:52.310230 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:52.310389 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:52.310362 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:53.600200 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:53.600012 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:53.600773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:53.600386 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:53.600866 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:53.600840 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:53.600988 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:53.600971 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:53.601068 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:53.601024 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:53.601173 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:53.601154 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:53.622052 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:53.622017 2544 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3aba6f38_a65a_43a8_86db_43d96ad371ee.slice/crio-dce6ce14ff247ca1a2addc2f4fea71f25583d6df0626de6265ff8aa2e203e581.scope\": RecentStats: unable to find data in memory cache]" Jun 12 17:11:53.783927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:53.783888 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="dce6ce14ff247ca1a2addc2f4fea71f25583d6df0626de6265ff8aa2e203e581" exitCode=0 Jun 12 17:11:53.784084 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:53.783977 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"dce6ce14ff247ca1a2addc2f4fea71f25583d6df0626de6265ff8aa2e203e581"} Jun 12 17:11:55.597418 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:55.597378 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:55.597418 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:55.597397 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:55.598374 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:55.597397 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:55.598374 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:55.597502 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:55.598374 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:55.597569 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:55.598374 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:55.597618 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:55.792609 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:55.792572 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="5bf4f43bd8269d6c41e068dfa0ca7c6b6a86d5a74cce8e5fb3f466459d9544a2" exitCode=0 Jun 12 17:11:55.792774 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:55.792651 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"5bf4f43bd8269d6c41e068dfa0ca7c6b6a86d5a74cce8e5fb3f466459d9544a2"} Jun 12 17:11:57.290553 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.290505 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:57.291037 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.290677 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 12 17:11:57.291037 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.290701 2544 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 12 17:11:57.291037 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.290710 2544 projected.go:196] Error preparing data for projected volume kube-api-access-bq42m for pod openshift-network-diagnostics/network-check-target-bcj25: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:57.291037 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.290762 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m podName:632ae094-973e-4dae-9691-51fe7176356a nodeName:}" failed. No retries permitted until 2026-06-12 17:12:29.290748341 +0000 UTC m=+66.163396428 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq42m" (UniqueName: "kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m") pod "network-check-target-bcj25" (UID: "632ae094-973e-4dae-9691-51fe7176356a") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 12 17:11:57.391215 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.391182 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:57.391410 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.391229 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:57.391410 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.391347 2544 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:57.391410 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.391351 2544 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:57.391410 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.391410 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs podName:2fe3599f-2dd4-4b8e-a4c9-573c9c893623 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:29.391394632 +0000 UTC m=+66.264042721 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs") pod "network-metrics-daemon-ws7n9" (UID: "2fe3599f-2dd4-4b8e-a4c9-573c9c893623") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 12 17:11:57.391544 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.391423 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret podName:40d265f2-894f-43a3-8643-36b1157cf081 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:29.391417513 +0000 UTC m=+66.264065600 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret") pod "global-pull-secret-syncer-p25tr" (UID: "40d265f2-894f-43a3-8643-36b1157cf081") : object "kube-system"/"original-pull-secret" not registered Jun 12 17:11:57.597601 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.597508 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:57.597768 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.597507 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:57.597768 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.597652 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:57.597768 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.597646 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ws7n9" podUID="2fe3599f-2dd4-4b8e-a4c9-573c9c893623" Jun 12 17:11:57.597913 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.597768 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p25tr" podUID="40d265f2-894f-43a3-8643-36b1157cf081" Jun 12 17:11:57.597913 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:57.597857 2544 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-bcj25" podUID="632ae094-973e-4dae-9691-51fe7176356a" Jun 12 17:11:57.799183 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.799147 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="d5f8baf9410aab6ab21cb63e4d62079c307053a94a5c4b7db6258fd0b1587931" exitCode=0 Jun 12 17:11:57.799388 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:57.799204 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"d5f8baf9410aab6ab21cb63e4d62079c307053a94a5c4b7db6258fd0b1587931"} Jun 12 17:11:58.438996 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.438962 2544 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-121.ec2.internal" event="NodeReady" Jun 12 17:11:58.439420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.439116 2544 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jun 12 17:11:58.485139 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.484801 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:11:58.505382 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.505346 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-rpt28"] Jun 12 17:11:58.505598 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.505462 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.507591 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.507562 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 12 17:11:58.507931 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.507905 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 12 17:11:58.508247 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.508231 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-244sl\"" Jun 12 17:11:58.508898 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.508874 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 12 17:11:58.516622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.516596 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 12 17:11:58.519405 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.519384 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-nxlld"] Jun 12 17:11:58.519512 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.519495 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.521568 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.521551 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-45dck\"" Jun 12 17:11:58.522268 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.522248 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 12 17:11:58.522829 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.522810 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 12 17:11:58.545134 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.545096 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:11:58.545134 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.545134 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rpt28"] Jun 12 17:11:58.545327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.545153 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxlld"] Jun 12 17:11:58.545327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.545226 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.547597 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.547572 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 12 17:11:58.547760 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.547599 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 12 17:11:58.547760 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.547677 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-vfkq5\"" Jun 12 17:11:58.547896 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.547852 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 12 17:11:58.598476 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598445 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.598476 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598480 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598503 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lm6z\" (UniqueName: \"kubernetes.io/projected/c79afb0f-1937-47de-aeb0-99600d87bd6c-kube-api-access-7lm6z\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598570 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598623 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598649 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598675 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79afb0f-1937-47de-aeb0-99600d87bd6c-tmp-dir\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598712 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.598742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598735 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.599024 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598764 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c79afb0f-1937-47de-aeb0-99600d87bd6c-config-volume\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.599024 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598795 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k86zq\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.599024 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.598892 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.699622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699367 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.699622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699598 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.699622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699621 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665d5681-1fe2-400a-8c15-53e4817e0327-cert\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699646 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699685 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lm6z\" (UniqueName: \"kubernetes.io/projected/c79afb0f-1937-47de-aeb0-99600d87bd6c-kube-api-access-7lm6z\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699709 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699745 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cl84b\" (UniqueName: \"kubernetes.io/projected/665d5681-1fe2-400a-8c15-53e4817e0327-kube-api-access-cl84b\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699775 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:58.699780 2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:58.699850 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls podName:c79afb0f-1937-47de-aeb0-99600d87bd6c nodeName:}" failed. No retries permitted until 2026-06-12 17:11:59.199829358 +0000 UTC m=+36.072477449 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls") pod "dns-default-rpt28" (UID: "c79afb0f-1937-47de-aeb0-99600d87bd6c") : secret "dns-default-metrics-tls" not found Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:58.699852 2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 17:11:58.699870 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:58.699868 2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64645596dd-2k82v: secret "image-registry-tls" not found Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699877 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:58.699898 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls podName:e34f9da6-3036-4d03-b2dd-9890114b07a4 nodeName:}" failed. No retries permitted until 2026-06-12 17:11:59.199888292 +0000 UTC m=+36.072536398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls") pod "image-registry-64645596dd-2k82v" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4") : secret "image-registry-tls" not found Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699926 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79afb0f-1937-47de-aeb0-99600d87bd6c-tmp-dir\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.699969 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700008 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700038 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c79afb0f-1937-47de-aeb0-99600d87bd6c-config-volume\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700055 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700341 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700082 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k86zq\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700361 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.700678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700581 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c79afb0f-1937-47de-aeb0-99600d87bd6c-tmp-dir\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.700807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.700697 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/c79afb0f-1937-47de-aeb0-99600d87bd6c-config-volume\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.701030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.701007 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.704021 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.703995 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.704021 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.703998 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.712454 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.712430 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lm6z\" (UniqueName: \"kubernetes.io/projected/c79afb0f-1937-47de-aeb0-99600d87bd6c-kube-api-access-7lm6z\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:58.721536 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.721463 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.721688 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.721560 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k86zq\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:58.800596 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.800559 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665d5681-1fe2-400a-8c15-53e4817e0327-cert\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.800831 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.800633 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cl84b\" (UniqueName: \"kubernetes.io/projected/665d5681-1fe2-400a-8c15-53e4817e0327-kube-api-access-cl84b\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.803806 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.803776 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/665d5681-1fe2-400a-8c15-53e4817e0327-cert\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.809429 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.809400 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cl84b\" (UniqueName: \"kubernetes.io/projected/665d5681-1fe2-400a-8c15-53e4817e0327-kube-api-access-cl84b\") pod \"ingress-canary-nxlld\" (UID: \"665d5681-1fe2-400a-8c15-53e4817e0327\") " pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:58.855422 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:58.855380 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-nxlld" Jun 12 17:11:59.045807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.045774 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-nxlld"] Jun 12 17:11:59.050729 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:11:59.050697 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod665d5681_1fe2_400a_8c15_53e4817e0327.slice/crio-432d1787b0548aebfc9f88be5d8cf7d86348f608805da68885c637d582419c79 WatchSource:0}: Error finding container 432d1787b0548aebfc9f88be5d8cf7d86348f608805da68885c637d582419c79: Status 404 returned error can't find the container with id 432d1787b0548aebfc9f88be5d8cf7d86348f608805da68885c637d582419c79 Jun 12 17:11:59.204496 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.204402 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:11:59.204496 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.204438 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:11:59.204724 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:59.204555 2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 17:11:59.204724 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:59.204566 2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64645596dd-2k82v: secret "image-registry-tls" not found Jun 12 17:11:59.204724 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:59.204569 2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 17:11:59.204724 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:59.204615 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls podName:e34f9da6-3036-4d03-b2dd-9890114b07a4 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:00.204602668 +0000 UTC m=+37.077250755 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls") pod "image-registry-64645596dd-2k82v" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4") : secret "image-registry-tls" not found Jun 12 17:11:59.204724 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:11:59.204628 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls podName:c79afb0f-1937-47de-aeb0-99600d87bd6c nodeName:}" failed. No retries permitted until 2026-06-12 17:12:00.204622574 +0000 UTC m=+37.077270660 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls") pod "dns-default-rpt28" (UID: "c79afb0f-1937-47de-aeb0-99600d87bd6c") : secret "dns-default-metrics-tls" not found Jun 12 17:11:59.602267 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.601466 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:11:59.602267 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.601494 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:11:59.602267 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.601473 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:11:59.604262 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604235 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 12 17:11:59.604420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604313 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvzdp\"" Jun 12 17:11:59.604420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604240 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 12 17:11:59.604609 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604591 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 12 17:11:59.604672 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604629 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 12 17:11:59.604840 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.604820 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7tt5\"" Jun 12 17:11:59.805776 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:11:59.805738 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxlld" event={"ID":"665d5681-1fe2-400a-8c15-53e4817e0327","Type":"ContainerStarted","Data":"432d1787b0548aebfc9f88be5d8cf7d86348f608805da68885c637d582419c79"} Jun 12 17:12:00.215510 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:00.215476 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:00.215510 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:00.215514 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:00.215773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:00.215634 2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 17:12:00.215773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:00.215672 2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 17:12:00.215773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:00.215684 2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64645596dd-2k82v: secret "image-registry-tls" not found Jun 12 17:12:00.215773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:00.215707 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls podName:c79afb0f-1937-47de-aeb0-99600d87bd6c nodeName:}" failed. No retries permitted until 2026-06-12 17:12:02.215686925 +0000 UTC m=+39.088335012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls") pod "dns-default-rpt28" (UID: "c79afb0f-1937-47de-aeb0-99600d87bd6c") : secret "dns-default-metrics-tls" not found Jun 12 17:12:00.215773 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:00.215723 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls podName:e34f9da6-3036-4d03-b2dd-9890114b07a4 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:02.215715266 +0000 UTC m=+39.088363353 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls") pod "image-registry-64645596dd-2k82v" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4") : secret "image-registry-tls" not found Jun 12 17:12:02.230724 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:02.230672 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:02.230724 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:02.230730 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:02.231234 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:02.230851 2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 17:12:02.231234 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:02.230873 2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 17:12:02.231234 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:02.230889 2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64645596dd-2k82v: secret "image-registry-tls" not found Jun 12 17:12:02.231234 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:02.230938 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls podName:c79afb0f-1937-47de-aeb0-99600d87bd6c nodeName:}" failed. No retries permitted until 2026-06-12 17:12:06.230915832 +0000 UTC m=+43.103563923 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls") pod "dns-default-rpt28" (UID: "c79afb0f-1937-47de-aeb0-99600d87bd6c") : secret "dns-default-metrics-tls" not found Jun 12 17:12:02.231234 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:02.230959 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls podName:e34f9da6-3036-4d03-b2dd-9890114b07a4 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:06.230949217 +0000 UTC m=+43.103597305 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls") pod "image-registry-64645596dd-2k82v" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4") : secret "image-registry-tls" not found Jun 12 17:12:03.310356 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.310325 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-s58z8"] Jun 12 17:12:03.329282 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.329254 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-s58z8"] Jun 12 17:12:03.329468 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.329371 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.332411 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.332383 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jun 12 17:12:03.333891 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.333874 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jun 12 17:12:03.334520 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.334500 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jun 12 17:12:03.334598 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.334501 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jun 12 17:12:03.334598 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.334501 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-ksjlr\"" Jun 12 17:12:03.441338 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.441227 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-cabundle\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.441507 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.441378 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-key\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.441560 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.441506 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cn7x\" (UniqueName: \"kubernetes.io/projected/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-kube-api-access-7cn7x\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.542061 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.542032 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-cabundle\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.542206 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.542126 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-key\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.542206 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.542190 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cn7x\" (UniqueName: \"kubernetes.io/projected/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-kube-api-access-7cn7x\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.542778 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.542756 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-cabundle\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.545850 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.545826 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-signing-key\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.552391 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.552361 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cn7x\" (UniqueName: \"kubernetes.io/projected/16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8-kube-api-access-7cn7x\") pod \"service-ca-7b994bc59-s58z8\" (UID: \"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8\") " pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.639011 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.638974 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7b994bc59-s58z8" Jun 12 17:12:03.801699 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.801671 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7b994bc59-s58z8"] Jun 12 17:12:03.819093 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.819065 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="19d10afbd82c91082ae1876030b326f3ff29ddd1c4c564dd74af8d033cbec080" exitCode=0 Jun 12 17:12:03.819235 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.819129 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"19d10afbd82c91082ae1876030b326f3ff29ddd1c4c564dd74af8d033cbec080"} Jun 12 17:12:03.821273 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:03.821250 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-nxlld" event={"ID":"665d5681-1fe2-400a-8c15-53e4817e0327","Type":"ContainerStarted","Data":"212898c5410b94d6ce4261c991a185ae7b50ae9444eac9f2e9445959831ce1a7"} Jun 12 17:12:03.822263 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:03.822236 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16f8b02d_13c5_4a44_ab0f_854cb8d7c5f8.slice/crio-21fb6d065c44c5b40f9e850ebd62efb559c1c4e1c4efa20169fbc11db96cc712 WatchSource:0}: Error finding container 21fb6d065c44c5b40f9e850ebd62efb559c1c4e1c4efa20169fbc11db96cc712: Status 404 returned error can't find the container with id 21fb6d065c44c5b40f9e850ebd62efb559c1c4e1c4efa20169fbc11db96cc712 Jun 12 17:12:04.004134 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:04.004029 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-nxlld" podStartSLOduration=1.872801544 podStartE2EDuration="6.004011171s" totalImagesPullingTime="4.131209627s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:04.003827515 +0000 UTC m=+40.876475623" watchObservedRunningTime="2026-06-12 17:12:04.004011171 +0000 UTC m=+40.876659270" Jun 12 17:12:04.827917 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:04.827878 2544 generic.go:356] "Generic (PLEG): container finished" podID="3aba6f38-a65a-43a8-86db-43d96ad371ee" containerID="d6cb13474052c3b252a6127ffe31fe8bdb70d170f1918e32a2016f2514f837f1" exitCode=0 Jun 12 17:12:04.828467 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:04.827964 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerDied","Data":"d6cb13474052c3b252a6127ffe31fe8bdb70d170f1918e32a2016f2514f837f1"} Jun 12 17:12:04.829863 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:04.829839 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-s58z8" event={"ID":"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8","Type":"ContainerStarted","Data":"21fb6d065c44c5b40f9e850ebd62efb559c1c4e1c4efa20169fbc11db96cc712"} Jun 12 17:12:05.836466 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:05.836433 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" event={"ID":"3aba6f38-a65a-43a8-86db-43d96ad371ee","Type":"ContainerStarted","Data":"606e7532270084e1334f66a234a2e7a8bf4d1cbb61d6f9afde0003166f5c9f1a"} Jun 12 17:12:05.882714 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:05.882659 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-dzhv9" podStartSLOduration=8.977944491 podStartE2EDuration="42.882644403s" totalImagesPullingTime="33.904699912s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:05.881817479 +0000 UTC m=+42.754465589" watchObservedRunningTime="2026-06-12 17:12:05.882644403 +0000 UTC m=+42.755292512" Jun 12 17:12:06.262383 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:06.262342 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:06.262383 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:06.262386 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:06.262591 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:06.262496 2544 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 12 17:12:06.262591 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:06.262521 2544 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 12 17:12:06.262591 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:06.262531 2544 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-64645596dd-2k82v: secret "image-registry-tls" not found Jun 12 17:12:06.262591 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:06.262562 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls podName:c79afb0f-1937-47de-aeb0-99600d87bd6c nodeName:}" failed. No retries permitted until 2026-06-12 17:12:14.262545886 +0000 UTC m=+51.135193973 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls") pod "dns-default-rpt28" (UID: "c79afb0f-1937-47de-aeb0-99600d87bd6c") : secret "dns-default-metrics-tls" not found Jun 12 17:12:06.262591 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:06.262576 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls podName:e34f9da6-3036-4d03-b2dd-9890114b07a4 nodeName:}" failed. No retries permitted until 2026-06-12 17:12:14.262570322 +0000 UTC m=+51.135218409 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls") pod "image-registry-64645596dd-2k82v" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4") : secret "image-registry-tls" not found Jun 12 17:12:06.843191 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:06.843143 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7b994bc59-s58z8" event={"ID":"16f8b02d-13c5-4a44-ab0f-854cb8d7c5f8","Type":"ContainerStarted","Data":"5f2d9e939e272921d7aae8c4e85d22752a4602f0b6f911f4aca4ae887d20e2a5"} Jun 12 17:12:06.870867 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:06.870806 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7b994bc59-s58z8" podStartSLOduration=1.700644541 podStartE2EDuration="3.870790291s" totalImagesPullingTime="2.17014575s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:06.870185342 +0000 UTC m=+43.742833452" watchObservedRunningTime="2026-06-12 17:12:06.870790291 +0000 UTC m=+43.743438400" Jun 12 17:12:14.330125 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.330074 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:14.330125 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.330123 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:14.332668 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.332634 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/c79afb0f-1937-47de-aeb0-99600d87bd6c-metrics-tls\") pod \"dns-default-rpt28\" (UID: \"c79afb0f-1937-47de-aeb0-99600d87bd6c\") " pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:14.332790 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.332728 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"image-registry-64645596dd-2k82v\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:14.416267 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.416225 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:14.428467 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.428425 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:14.638880 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.638844 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:12:14.642457 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:14.642429 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode34f9da6_3036_4d03_b2dd_9890114b07a4.slice/crio-a141ec2b1bc4e8d10eeb907a3330aec6108cf021a253ecb172db372588dee673 WatchSource:0}: Error finding container a141ec2b1bc4e8d10eeb907a3330aec6108cf021a253ecb172db372588dee673: Status 404 returned error can't find the container with id a141ec2b1bc4e8d10eeb907a3330aec6108cf021a253ecb172db372588dee673 Jun 12 17:12:14.654956 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.654918 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-rpt28"] Jun 12 17:12:14.659044 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:14.659015 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc79afb0f_1937_47de_aeb0_99600d87bd6c.slice/crio-5e3d8f0f93fc80b2e677fcac938d883189e217deaf3578c53340b15d725065cf WatchSource:0}: Error finding container 5e3d8f0f93fc80b2e677fcac938d883189e217deaf3578c53340b15d725065cf: Status 404 returned error can't find the container with id 5e3d8f0f93fc80b2e677fcac938d883189e217deaf3578c53340b15d725065cf Jun 12 17:12:14.863566 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.863514 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpt28" event={"ID":"c79afb0f-1937-47de-aeb0-99600d87bd6c","Type":"ContainerStarted","Data":"5e3d8f0f93fc80b2e677fcac938d883189e217deaf3578c53340b15d725065cf"} Jun 12 17:12:14.864891 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.864862 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64645596dd-2k82v" event={"ID":"e34f9da6-3036-4d03-b2dd-9890114b07a4","Type":"ContainerStarted","Data":"e0365bdd40cbfc35add31b960adaa6f06c1fd8366513f11137a9339ecb9e109d"} Jun 12 17:12:14.865039 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.864903 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64645596dd-2k82v" event={"ID":"e34f9da6-3036-4d03-b2dd-9890114b07a4","Type":"ContainerStarted","Data":"a141ec2b1bc4e8d10eeb907a3330aec6108cf021a253ecb172db372588dee673"} Jun 12 17:12:14.865039 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.864990 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:14.888529 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:14.888476 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-64645596dd-2k82v" podStartSLOduration=36.888460294 podStartE2EDuration="36.888460294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:14.887713627 +0000 UTC m=+51.760361780" watchObservedRunningTime="2026-06-12 17:12:14.888460294 +0000 UTC m=+51.761108399" Jun 12 17:12:17.876649 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:17.876606 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpt28" event={"ID":"c79afb0f-1937-47de-aeb0-99600d87bd6c","Type":"ContainerStarted","Data":"08f68b9e3547f760502c3e3a58b62c3c12f6889e584d8b4b1ca0f4ceabf62eba"} Jun 12 17:12:17.877044 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:17.876688 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-rpt28" event={"ID":"c79afb0f-1937-47de-aeb0-99600d87bd6c","Type":"ContainerStarted","Data":"23972623100bf130a3d2d23ae10de2a08351286b4a26eb95f48dbd26a1c1d529"} Jun 12 17:12:17.877044 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:17.876760 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:17.902377 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:17.902321 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-rpt28" podStartSLOduration=17.11765422 podStartE2EDuration="19.902287973s" totalImagesPullingTime="2.784633753s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:17.901711086 +0000 UTC m=+54.774359193" watchObservedRunningTime="2026-06-12 17:12:17.902287973 +0000 UTC m=+54.774936059" Jun 12 17:12:22.796089 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:22.796056 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-69swk" Jun 12 17:12:27.478509 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.478471 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b"] Jun 12 17:12:27.480937 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.480910 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5859ddd786-h5s8d"] Jun 12 17:12:27.481185 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.481044 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.482890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.482869 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.487156 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.487129 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jun 12 17:12:27.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.488701 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jun 12 17:12:27.488818 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.488701 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jun 12 17:12:27.489313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489276 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 12 17:12:27.489426 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489352 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-2g2q8\"" Jun 12 17:12:27.489502 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489488 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 12 17:12:27.489596 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489578 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 12 17:12:27.489676 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489660 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 12 17:12:27.489750 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489704 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 12 17:12:27.489750 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489734 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 12 17:12:27.489861 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.489796 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jun 12 17:12:27.502076 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.502047 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b"] Jun 12 17:12:27.520246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.520210 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:12:27.521393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.521366 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5859ddd786-h5s8d"] Jun 12 17:12:27.528126 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528098 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-stats-auth\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.528126 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528134 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-metrics-certs\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.528327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528195 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-default-certificate\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.528327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528222 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdsg5\" (UniqueName: \"kubernetes.io/projected/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-kube-api-access-tdsg5\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.528327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528269 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-tmp\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.528327 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528322 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4drh\" (UniqueName: \"kubernetes.io/projected/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-kube-api-access-t4drh\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.528465 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528348 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-klusterlet-config\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.528465 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.528419 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-service-ca-bundle\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.629405 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629368 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-stats-auth\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.629405 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629401 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-metrics-certs\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.629616 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629423 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-default-certificate\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.629616 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629568 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdsg5\" (UniqueName: \"kubernetes.io/projected/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-kube-api-access-tdsg5\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.629616 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629612 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-tmp\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.629762 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629638 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4drh\" (UniqueName: \"kubernetes.io/projected/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-kube-api-access-t4drh\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.629762 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629671 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-klusterlet-config\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.629762 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.629715 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-service-ca-bundle\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.630056 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.630033 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-tmp\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.630404 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.630380 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-service-ca-bundle\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.632171 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.632121 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-stats-auth\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.632491 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.632471 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-metrics-certs\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.632491 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.632482 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-klusterlet-config\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.632629 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.632607 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-default-certificate\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.642520 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.642493 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdsg5\" (UniqueName: \"kubernetes.io/projected/5d2dc626-325e-41ff-b854-bf4c9ce12ea7-kube-api-access-tdsg5\") pod \"router-default-5859ddd786-h5s8d\" (UID: \"5d2dc626-325e-41ff-b854-bf4c9ce12ea7\") " pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.643498 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.643475 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4drh\" (UniqueName: \"kubernetes.io/projected/38631e7b-afcd-4c3b-a3cf-b8ff45a9048e-kube-api-access-t4drh\") pod \"klusterlet-addon-workmgr-77df449449-j5v6b\" (UID: \"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.717574 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.717539 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5df4d4978c-5t5q2"] Jun 12 17:12:27.720395 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.720372 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.747344 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.747247 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5df4d4978c-5t5q2"] Jun 12 17:12:27.792020 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.791980 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:27.796842 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.796821 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831556 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-certificates\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831602 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-installation-pull-secrets\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831653 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-image-registry-private-configuration\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831682 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-bound-sa-token\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831708 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mkgwq\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-kube-api-access-mkgwq\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831735 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-tls\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831825 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bd6c12a-724c-4049-b22f-771b7a07b33d-ca-trust-extracted\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.832030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.831866 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-trusted-ca\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.881793 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.881766 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-rpt28" Jun 12 17:12:27.932512 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.932471 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-image-registry-private-configuration\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.932721 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.932523 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-bound-sa-token\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.932721 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.932550 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mkgwq\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-kube-api-access-mkgwq\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.932843 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.932733 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-tls\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.933502 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.933007 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bd6c12a-724c-4049-b22f-771b7a07b33d-ca-trust-extracted\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.933502 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.933066 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-trusted-ca\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.933502 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.933117 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-certificates\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.933502 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.933144 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-installation-pull-secrets\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.933781 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.933667 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2bd6c12a-724c-4049-b22f-771b7a07b33d-ca-trust-extracted\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.934794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.934766 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-trusted-ca\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.934961 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.934896 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-certificates\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.936366 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.936322 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-registry-tls\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.936366 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.936344 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-image-registry-private-configuration\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.937067 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.936995 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2bd6c12a-724c-4049-b22f-771b7a07b33d-installation-pull-secrets\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.945208 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.945174 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b"] Jun 12 17:12:27.946979 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.946958 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mkgwq\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-kube-api-access-mkgwq\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.948557 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.948534 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2bd6c12a-724c-4049-b22f-771b7a07b33d-bound-sa-token\") pod \"image-registry-5df4d4978c-5t5q2\" (UID: \"2bd6c12a-724c-4049-b22f-771b7a07b33d\") " pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:27.948672 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:27.948538 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38631e7b_afcd_4c3b_a3cf_b8ff45a9048e.slice/crio-947dccb269883d44ede6ce1b604f51464a162c038abd32a3c40a34d2dc0ee7d7 WatchSource:0}: Error finding container 947dccb269883d44ede6ce1b604f51464a162c038abd32a3c40a34d2dc0ee7d7: Status 404 returned error can't find the container with id 947dccb269883d44ede6ce1b604f51464a162c038abd32a3c40a34d2dc0ee7d7 Jun 12 17:12:27.963372 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:27.963340 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5859ddd786-h5s8d"] Jun 12 17:12:27.966960 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:27.966931 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d2dc626_325e_41ff_b854_bf4c9ce12ea7.slice/crio-1719f77158ca25187eed3b2eb474554ec0be05c15cd665982b6b99d359a33b64 WatchSource:0}: Error finding container 1719f77158ca25187eed3b2eb474554ec0be05c15cd665982b6b99d359a33b64: Status 404 returned error can't find the container with id 1719f77158ca25187eed3b2eb474554ec0be05c15cd665982b6b99d359a33b64 Jun 12 17:12:28.029520 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.029489 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:28.182137 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.182101 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5df4d4978c-5t5q2"] Jun 12 17:12:28.184981 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:28.184943 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd6c12a_724c_4049_b22f_771b7a07b33d.slice/crio-109154d24258291e75ffed91144545fd4a8ca8ef3df99cf229ced7b6208e98ed WatchSource:0}: Error finding container 109154d24258291e75ffed91144545fd4a8ca8ef3df99cf229ced7b6208e98ed: Status 404 returned error can't find the container with id 109154d24258291e75ffed91144545fd4a8ca8ef3df99cf229ced7b6208e98ed Jun 12 17:12:28.910079 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.910041 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" event={"ID":"2bd6c12a-724c-4049-b22f-771b7a07b33d","Type":"ContainerStarted","Data":"33738a35b1593e0018cbf9f3601140dd25cdb68b6a76e6022010d1ffa8e29ba9"} Jun 12 17:12:28.910579 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.910092 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" event={"ID":"2bd6c12a-724c-4049-b22f-771b7a07b33d","Type":"ContainerStarted","Data":"109154d24258291e75ffed91144545fd4a8ca8ef3df99cf229ced7b6208e98ed"} Jun 12 17:12:28.910579 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.910135 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:28.911736 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.911712 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5859ddd786-h5s8d" event={"ID":"5d2dc626-325e-41ff-b854-bf4c9ce12ea7","Type":"ContainerStarted","Data":"ddc487e969283916564e4fe6d4ad6c1b52dfc5bd07cc6c293586d61cfa608869"} Jun 12 17:12:28.911833 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.911759 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5859ddd786-h5s8d" event={"ID":"5d2dc626-325e-41ff-b854-bf4c9ce12ea7","Type":"ContainerStarted","Data":"1719f77158ca25187eed3b2eb474554ec0be05c15cd665982b6b99d359a33b64"} Jun 12 17:12:28.912753 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.912730 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" event={"ID":"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e","Type":"ContainerStarted","Data":"947dccb269883d44ede6ce1b604f51464a162c038abd32a3c40a34d2dc0ee7d7"} Jun 12 17:12:28.930207 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.930151 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" podStartSLOduration=1.930134901 podStartE2EDuration="1.930134901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:28.929280557 +0000 UTC m=+65.801928667" watchObservedRunningTime="2026-06-12 17:12:28.930134901 +0000 UTC m=+65.802783010" Jun 12 17:12:28.950725 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:28.950675 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5859ddd786-h5s8d" podStartSLOduration=1.950657702 podStartE2EDuration="1.950657702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:28.949739172 +0000 UTC m=+65.822387304" watchObservedRunningTime="2026-06-12 17:12:28.950657702 +0000 UTC m=+65.823305790" Jun 12 17:12:29.344414 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.344311 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:12:29.347133 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.347088 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 12 17:12:29.358289 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.358002 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 12 17:12:29.369030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.368964 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq42m\" (UniqueName: \"kubernetes.io/projected/632ae094-973e-4dae-9691-51fe7176356a-kube-api-access-bq42m\") pod \"network-check-target-bcj25\" (UID: \"632ae094-973e-4dae-9691-51fe7176356a\") " pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:12:29.444734 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.444684 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:12:29.444904 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.444751 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:12:29.447973 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.447945 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 12 17:12:29.448165 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.448135 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 12 17:12:29.458012 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.457969 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/40d265f2-894f-43a3-8643-36b1157cf081-original-pull-secret\") pod \"global-pull-secret-syncer-p25tr\" (UID: \"40d265f2-894f-43a3-8643-36b1157cf081\") " pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:12:29.461153 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.461127 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2fe3599f-2dd4-4b8e-a4c9-573c9c893623-metrics-certs\") pod \"network-metrics-daemon-ws7n9\" (UID: \"2fe3599f-2dd4-4b8e-a4c9-573c9c893623\") " pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:12:29.617034 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.616998 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-hvzdp\"" Jun 12 17:12:29.624031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.623991 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-l7tt5\"" Jun 12 17:12:29.625167 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.624942 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:12:29.628418 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.628203 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p25tr" Jun 12 17:12:29.632929 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.632901 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ws7n9" Jun 12 17:12:29.797223 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.797192 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:29.800023 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.799771 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:29.829852 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.829798 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-bcj25"] Jun 12 17:12:29.846211 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.846171 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p25tr"] Jun 12 17:12:29.850976 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:29.850937 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40d265f2_894f_43a3_8643_36b1157cf081.slice/crio-4cb3f0042530e1ad80dbcb6637df3eb66c30b1f9d06376d8e367abbde54f0bae WatchSource:0}: Error finding container 4cb3f0042530e1ad80dbcb6637df3eb66c30b1f9d06376d8e367abbde54f0bae: Status 404 returned error can't find the container with id 4cb3f0042530e1ad80dbcb6637df3eb66c30b1f9d06376d8e367abbde54f0bae Jun 12 17:12:29.865615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.865580 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ws7n9"] Jun 12 17:12:29.870688 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:29.870633 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fe3599f_2dd4_4b8e_a4c9_573c9c893623.slice/crio-57b872874d70519a7186f59d786904ac8432d73acd34996e8e218e8d9545f681 WatchSource:0}: Error finding container 57b872874d70519a7186f59d786904ac8432d73acd34996e8e218e8d9545f681: Status 404 returned error can't find the container with id 57b872874d70519a7186f59d786904ac8432d73acd34996e8e218e8d9545f681 Jun 12 17:12:29.917845 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.917767 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p25tr" event={"ID":"40d265f2-894f-43a3-8643-36b1157cf081","Type":"ContainerStarted","Data":"4cb3f0042530e1ad80dbcb6637df3eb66c30b1f9d06376d8e367abbde54f0bae"} Jun 12 17:12:29.919256 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.919222 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ws7n9" event={"ID":"2fe3599f-2dd4-4b8e-a4c9-573c9c893623","Type":"ContainerStarted","Data":"57b872874d70519a7186f59d786904ac8432d73acd34996e8e218e8d9545f681"} Jun 12 17:12:29.922017 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.921907 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcj25" event={"ID":"632ae094-973e-4dae-9691-51fe7176356a","Type":"ContainerStarted","Data":"b44f61f7128271d408bce3522f0b8f7496ccd29e6c950be456576645eb9a096e"} Jun 12 17:12:29.922690 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.922665 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:29.924010 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:29.923981 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5859ddd786-h5s8d" Jun 12 17:12:33.940338 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.940276 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" event={"ID":"38631e7b-afcd-4c3b-a3cf-b8ff45a9048e","Type":"ContainerStarted","Data":"5903f86425041e483a9ea4a99a22047776b9437e25c8a7472f732af570846627"} Jun 12 17:12:33.940816 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.940470 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:33.942397 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.942358 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" Jun 12 17:12:33.942798 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.942772 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ws7n9" event={"ID":"2fe3599f-2dd4-4b8e-a4c9-573c9c893623","Type":"ContainerStarted","Data":"dbf587b5dbbdfae49f2cecfb396ac155bef01031932c85e86e05f293c72e7ad7"} Jun 12 17:12:33.942912 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.942818 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ws7n9" event={"ID":"2fe3599f-2dd4-4b8e-a4c9-573c9c893623","Type":"ContainerStarted","Data":"edaf6c8779fae4a2adfde518564e52581e2f5f116050f7745c4b26341a0c410b"} Jun 12 17:12:33.959268 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.959202 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-77df449449-j5v6b" podStartSLOduration=1.7431263590000001 podStartE2EDuration="6.959181561s" totalImagesPullingTime="5.216055202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:33.958287853 +0000 UTC m=+70.830935986" watchObservedRunningTime="2026-06-12 17:12:33.959181561 +0000 UTC m=+70.831829670" Jun 12 17:12:33.974919 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:33.974701 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ws7n9" podStartSLOduration=67.695425665 podStartE2EDuration="1m10.974657798s" totalImagesPullingTime="3.279232133s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:33.974227032 +0000 UTC m=+70.846875147" watchObservedRunningTime="2026-06-12 17:12:33.974657798 +0000 UTC m=+70.847305915" Jun 12 17:12:34.848494 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.848452 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5sgrm"] Jun 12 17:12:34.850807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.850784 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.852946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.852915 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 12 17:12:34.853099 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.852979 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 12 17:12:34.853099 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.852923 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 12 17:12:34.853724 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.853705 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6z56\"" Jun 12 17:12:34.853838 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.853726 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 12 17:12:34.863690 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.863664 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5sgrm"] Jun 12 17:12:34.892418 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.892384 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.892603 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.892438 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/14df4b50-7be7-4b74-aec5-e6fd2430074f-data-volume\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.892603 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.892501 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr9xv\" (UniqueName: \"kubernetes.io/projected/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-api-access-rr9xv\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.892603 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.892557 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.892742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.892613 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/14df4b50-7be7-4b74-aec5-e6fd2430074f-crio-socket\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.993800 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.993760 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.993828 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/14df4b50-7be7-4b74-aec5-e6fd2430074f-data-volume\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.993856 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr9xv\" (UniqueName: \"kubernetes.io/projected/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-api-access-rr9xv\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.993891 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:34.993930 2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.993994 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/14df4b50-7be7-4b74-aec5-e6fd2430074f-crio-socket\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994260 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:34.994011 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:35.493988737 +0000 UTC m=+72.366636842 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : secret "insights-runtime-extractor-tls" not found Jun 12 17:12:34.994510 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:34.994398 2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 17:12:34.994510 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:34.994465 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:35.494440514 +0000 UTC m=+72.367088628 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : configmap "kube-rbac-proxy" not found Jun 12 17:12:34.994510 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.994459 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/14df4b50-7be7-4b74-aec5-e6fd2430074f-crio-socket\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:34.994707 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:34.994684 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/14df4b50-7be7-4b74-aec5-e6fd2430074f-data-volume\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:35.002782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:35.002752 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr9xv\" (UniqueName: \"kubernetes.io/projected/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-api-access-rr9xv\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:35.497668 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:35.497622 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:35.497882 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:35.497712 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:35.497882 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:35.497833 2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 17:12:35.498068 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:35.497899 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:36.497881316 +0000 UTC m=+73.370529404 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : configmap "kube-rbac-proxy" not found Jun 12 17:12:35.498365 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:35.498345 2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 17:12:35.498451 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:35.498406 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:36.498389908 +0000 UTC m=+73.371038016 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : secret "insights-runtime-extractor-tls" not found Jun 12 17:12:36.282861 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.282826 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpt28_c79afb0f-1937-47de-aeb0-99600d87bd6c/dns/0.log" Jun 12 17:12:36.454640 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.454611 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpt28_c79afb0f-1937-47de-aeb0-99600d87bd6c/kube-rbac-proxy/0.log" Jun 12 17:12:36.506940 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.506889 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:36.507137 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.506969 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:36.507137 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:36.507056 2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 17:12:36.507137 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:36.507061 2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 17:12:36.507137 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:36.507111 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:38.507098839 +0000 UTC m=+75.379746926 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : configmap "kube-rbac-proxy" not found Jun 12 17:12:36.507137 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:36.507138 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:38.507120556 +0000 UTC m=+75.379768651 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : secret "insights-runtime-extractor-tls" not found Jun 12 17:12:36.865127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.865100 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p25h9_e159f176-bd80-4a5a-93c2-abcbd609ec4f/dns-node-resolver/0.log" Jun 12 17:12:36.954468 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.954429 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p25tr" event={"ID":"40d265f2-894f-43a3-8643-36b1157cf081","Type":"ContainerStarted","Data":"08ba0f8b331b8f6e6bfaca33ffa4f60e92f654fca6f11f40b5803ac4da7e85c4"} Jun 12 17:12:36.955872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.955839 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-bcj25" event={"ID":"632ae094-973e-4dae-9691-51fe7176356a","Type":"ContainerStarted","Data":"4b5f55cc8b1fae4a1c51b555d1fd38431d4304bc7fc32b01c4def2a1836ab4f9"} Jun 12 17:12:36.956036 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.955959 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:12:36.974550 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.974493 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p25tr" podStartSLOduration=67.758391166 podStartE2EDuration="1m13.974478118s" totalImagesPullingTime="6.216086952s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:36.973787129 +0000 UTC m=+73.846435240" watchObservedRunningTime="2026-06-12 17:12:36.974478118 +0000 UTC m=+73.847126227" Jun 12 17:12:36.994131 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:36.994071 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-bcj25" podStartSLOduration=67.759468661 podStartE2EDuration="1m13.994057497s" totalImagesPullingTime="6.234588836s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:11:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:36.993882637 +0000 UTC m=+73.866530773" watchObservedRunningTime="2026-06-12 17:12:36.994057497 +0000 UTC m=+73.866705607" Jun 12 17:12:37.253733 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:37.253651 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5df4d4978c-5t5q2_2bd6c12a-724c-4049-b22f-771b7a07b33d/registry/0.log" Jun 12 17:12:37.458026 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:37.457995 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-64645596dd-2k82v_e34f9da6-3036-4d03-b2dd-9890114b07a4/registry/0.log" Jun 12 17:12:37.526604 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:37.526523 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:37.857200 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:37.857173 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hgck_028bb6d9-6629-443c-a31d-83abf2688151/node-ca/0.log" Jun 12 17:12:38.458798 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:38.458767 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:12:38.525262 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:38.525231 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:38.525448 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:38.525288 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:38.525448 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:38.525424 2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 17:12:38.525532 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:38.525444 2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 17:12:38.525532 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:38.525484 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:42.525466526 +0000 UTC m=+79.398114614 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : configmap "kube-rbac-proxy" not found Jun 12 17:12:38.525532 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:38.525508 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:42.525490059 +0000 UTC m=+79.398138149 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : secret "insights-runtime-extractor-tls" not found Jun 12 17:12:39.053752 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:39.053727 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxlld_665d5681-1fe2-400a-8c15-53e4817e0327/serve-healthcheck-canary/0.log" Jun 12 17:12:42.557494 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.557360 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:42.557494 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.557435 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:42.557890 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:42.557490 2544 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jun 12 17:12:42.557890 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:42.557544 2544 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 12 17:12:42.557890 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:42.557590 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:50.557569178 +0000 UTC m=+87.430217265 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : configmap "kube-rbac-proxy" not found Jun 12 17:12:42.557890 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:12:42.557611 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls podName:14df4b50-7be7-4b74-aec5-e6fd2430074f nodeName:}" failed. No retries permitted until 2026-06-12 17:12:50.55760134 +0000 UTC m=+87.430249428 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5sgrm" (UID: "14df4b50-7be7-4b74-aec5-e6fd2430074f") : secret "insights-runtime-extractor-tls" not found Jun 12 17:12:42.790683 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.790642 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-l9jmw"] Jun 12 17:12:42.794965 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.794942 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.800075 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800048 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 12 17:12:42.800228 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800051 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 12 17:12:42.800584 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800362 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 12 17:12:42.800584 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800423 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 12 17:12:42.800584 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800437 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-tmhrq\"" Jun 12 17:12:42.800584 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800454 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 12 17:12:42.800584 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.800365 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 12 17:12:42.860968 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.860935 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-tls\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.860981 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-root\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861006 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpm6h\" (UniqueName: \"kubernetes.io/projected/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-kube-api-access-wpm6h\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861061 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-sys\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861095 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861144 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861122 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-textfile\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861373 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861152 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861373 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861314 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-wtmp\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.861373 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.861356 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-metrics-client-ca\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962104 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962060 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-sys\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962104 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962105 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962132 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-textfile\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962175 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-sys\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962222 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962316 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-wtmp\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962344 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-metrics-client-ca\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962397 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-tls\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962439 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-root\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962459 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpm6h\" (UniqueName: \"kubernetes.io/projected/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-kube-api-access-wpm6h\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962512 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-textfile\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962519 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-wtmp\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962577 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-root\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.962816 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.962748 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-accelerators-collector-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.963204 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.963171 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-metrics-client-ca\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.965041 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.965018 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-tls\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.965208 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.965190 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:42.971585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:42.971556 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpm6h\" (UniqueName: \"kubernetes.io/projected/1b1e37dc-9346-42f0-b844-0eca9a1a1bd2-kube-api-access-wpm6h\") pod \"node-exporter-l9jmw\" (UID: \"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2\") " pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:43.105723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:43.105687 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-l9jmw" Jun 12 17:12:43.120882 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:43.120845 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1e37dc_9346_42f0_b844_0eca9a1a1bd2.slice/crio-0c871a1c3fcb9196fbc32f0638db38a84df90994d92ac031f37fb93cb9f252df WatchSource:0}: Error finding container 0c871a1c3fcb9196fbc32f0638db38a84df90994d92ac031f37fb93cb9f252df: Status 404 returned error can't find the container with id 0c871a1c3fcb9196fbc32f0638db38a84df90994d92ac031f37fb93cb9f252df Jun 12 17:12:43.981730 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:43.981677 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l9jmw" event={"ID":"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2","Type":"ContainerStarted","Data":"0c871a1c3fcb9196fbc32f0638db38a84df90994d92ac031f37fb93cb9f252df"} Jun 12 17:12:44.809461 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.809427 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-c65cbb6b-gwbb5"] Jun 12 17:12:44.812715 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.812697 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.817952 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.817924 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jun 12 17:12:44.818695 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.818618 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jun 12 17:12:44.818975 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.818755 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jun 12 17:12:44.818975 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.818848 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-dmvmb\"" Jun 12 17:12:44.819094 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.819076 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jun 12 17:12:44.819149 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.819116 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-dspoadmsb368i\"" Jun 12 17:12:44.819276 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.819251 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jun 12 17:12:44.827727 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.827702 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c65cbb6b-gwbb5"] Jun 12 17:12:44.878365 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878327 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc155ca-673e-48a4-801b-dd081e8a099c-metrics-client-ca\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878365 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878365 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4vwp\" (UniqueName: \"kubernetes.io/projected/ecc155ca-673e-48a4-801b-dd081e8a099c-kube-api-access-l4vwp\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878425 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878449 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878473 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-grpc-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878560 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878710 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878587 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.878710 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.878648 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.979813 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979780 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-grpc-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.979959 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979838 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.979959 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979863 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.979959 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979927 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.980081 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979975 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc155ca-673e-48a4-801b-dd081e8a099c-metrics-client-ca\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.980081 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.979997 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4vwp\" (UniqueName: \"kubernetes.io/projected/ecc155ca-673e-48a4-801b-dd081e8a099c-kube-api-access-l4vwp\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.980081 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.980043 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.980217 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.980134 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.980958 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.980828 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ecc155ca-673e-48a4-801b-dd081e8a099c-metrics-client-ca\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983032 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983002 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983477 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983047 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983477 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983100 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983477 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983201 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983477 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983440 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-grpc-tls\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.983688 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.983606 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ecc155ca-673e-48a4-801b-dd081e8a099c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:44.986869 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.986843 2544 generic.go:356] "Generic (PLEG): container finished" podID="1b1e37dc-9346-42f0-b844-0eca9a1a1bd2" containerID="ec89cf762235c1a5270f5c2657e694456efb85af5145e0dee56ac2e6797d0b36" exitCode=0 Jun 12 17:12:44.986961 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.986921 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l9jmw" event={"ID":"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2","Type":"ContainerDied","Data":"ec89cf762235c1a5270f5c2657e694456efb85af5145e0dee56ac2e6797d0b36"} Jun 12 17:12:44.990374 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:44.990353 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4vwp\" (UniqueName: \"kubernetes.io/projected/ecc155ca-673e-48a4-801b-dd081e8a099c-kube-api-access-l4vwp\") pod \"thanos-querier-c65cbb6b-gwbb5\" (UID: \"ecc155ca-673e-48a4-801b-dd081e8a099c\") " pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:45.122183 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:45.122144 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:45.266578 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:45.266534 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-c65cbb6b-gwbb5"] Jun 12 17:12:45.270782 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:45.270753 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecc155ca_673e_48a4_801b_dd081e8a099c.slice/crio-2500f6bfc91b6dbb0ebb2f8e4b778634d4d83068273e3364eac02e3b06575051 WatchSource:0}: Error finding container 2500f6bfc91b6dbb0ebb2f8e4b778634d4d83068273e3364eac02e3b06575051: Status 404 returned error can't find the container with id 2500f6bfc91b6dbb0ebb2f8e4b778634d4d83068273e3364eac02e3b06575051 Jun 12 17:12:45.993106 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:45.993062 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l9jmw" event={"ID":"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2","Type":"ContainerStarted","Data":"84e13aeaedf3ed97f9e5015ba326821f3bb149cb3624bf7f839b5d6d57d9a0ee"} Jun 12 17:12:45.993621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:45.993125 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-l9jmw" event={"ID":"1b1e37dc-9346-42f0-b844-0eca9a1a1bd2","Type":"ContainerStarted","Data":"c86e19dbd5dfc3c75471576b536955b8b7ace88b20f3931da34e7c2c2b5afad8"} Jun 12 17:12:45.994243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:45.994222 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"2500f6bfc91b6dbb0ebb2f8e4b778634d4d83068273e3364eac02e3b06575051"} Jun 12 17:12:46.012585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:46.012532 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-l9jmw" podStartSLOduration=2.805466219 podStartE2EDuration="4.012515393s" totalImagesPullingTime="1.134135946s" totalInitContainerRuntime="72.913228ms" isStatefulPod=true podCreationTimestamp="2026-06-12 17:12:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:46.010745173 +0000 UTC m=+82.883393304" watchObservedRunningTime="2026-06-12 17:12:46.012515393 +0000 UTC m=+82.885163501" Jun 12 17:12:47.258668 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.258631 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5b5ff785fc-jmv2h"] Jun 12 17:12:47.261763 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.261745 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.264412 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.264381 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 12 17:12:47.265064 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.265041 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-v54vt\"" Jun 12 17:12:47.265221 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.265073 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 12 17:12:47.265221 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.265131 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 12 17:12:47.265221 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.265161 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-88bmatr3moo4s\"" Jun 12 17:12:47.265221 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.265174 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 12 17:12:47.289898 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.289858 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b5ff785fc-jmv2h"] Jun 12 17:12:47.404580 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404546 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-tls\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.404758 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404586 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.404758 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404624 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kl7fb\" (UniqueName: \"kubernetes.io/projected/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-kube-api-access-kl7fb\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.404758 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404700 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-client-certs\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.404911 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404800 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-audit-log\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.404911 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404872 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-metrics-server-audit-profiles\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.405002 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.404907 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-client-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506120 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-tls\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506162 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506199 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kl7fb\" (UniqueName: \"kubernetes.io/projected/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-kube-api-access-kl7fb\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506229 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-client-certs\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506258 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-audit-log\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.506894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.506281 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-metrics-server-audit-profiles\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.509704 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.509526 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-client-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.509704 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.507501 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-audit-log\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.509704 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.509197 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.509704 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.508841 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-metrics-server-audit-profiles\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.512755 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.512722 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-client-ca-bundle\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.515518 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.515466 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kl7fb\" (UniqueName: \"kubernetes.io/projected/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-kube-api-access-kl7fb\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.516432 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.516400 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-client-certs\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.518264 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.518240 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/8b28f84f-ec7e-4077-87f1-7ebb168b8e7f-secret-metrics-server-tls\") pod \"metrics-server-5b5ff785fc-jmv2h\" (UID: \"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f\") " pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:47.571661 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:47.571627 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:12:48.080878 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:48.080847 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5b5ff785fc-jmv2h"] Jun 12 17:12:48.084737 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:48.084704 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b28f84f_ec7e_4077_87f1_7ebb168b8e7f.slice/crio-2cc6d5a080796fad14324a949bb91a78db2eac10a3bdc8ada83957a32925295b WatchSource:0}: Error finding container 2cc6d5a080796fad14324a949bb91a78db2eac10a3bdc8ada83957a32925295b: Status 404 returned error can't find the container with id 2cc6d5a080796fad14324a949bb91a78db2eac10a3bdc8ada83957a32925295b Jun 12 17:12:49.010037 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.009996 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" event={"ID":"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f","Type":"ContainerStarted","Data":"2cc6d5a080796fad14324a949bb91a78db2eac10a3bdc8ada83957a32925295b"} Jun 12 17:12:49.012330 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.012283 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"ec977fdf54d6648124493a94708ee9412de58a1ca9f5febcabc52d9adb9e41cf"} Jun 12 17:12:49.012447 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.012376 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"0817073e9acc96cc572df4fe7414f816efdc578e9e89bc9098070c2a76ab5e8d"} Jun 12 17:12:49.012447 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.012409 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"53a771ff07c1d8919a8e579d54c9a17337b4be8d53ef7ab05745c007ca7fd31f"} Jun 12 17:12:49.088003 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.087966 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:12:49.092674 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.092646 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.095918 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.095888 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 12 17:12:49.096070 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.095899 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 12 17:12:49.096173 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096147 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 12 17:12:49.096259 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096206 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 12 17:12:49.096349 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096278 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 12 17:12:49.096615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096444 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 12 17:12:49.096615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096441 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 12 17:12:49.096615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096449 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 12 17:12:49.096615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096565 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rb85v\"" Jun 12 17:12:49.096886 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096720 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 12 17:12:49.096942 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.096910 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 12 17:12:49.097892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.097202 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4497pcg7og20c\"" Jun 12 17:12:49.097892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.097215 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 12 17:12:49.097892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.097257 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 12 17:12:49.097892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.097372 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 12 17:12:49.100047 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.100008 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 12 17:12:49.111266 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.111239 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:12:49.225926 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.225894 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226057 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.225943 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226057 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.225986 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226057 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226015 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226057 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226041 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226068 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226098 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226146 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226176 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226216 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226252 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226245 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226562 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226272 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226562 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226372 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226562 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226411 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226562 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226475 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226562 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226510 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226844 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226587 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226844 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226640 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmh98\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226844 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226666 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.226844 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.226789 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327519 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327485 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327519 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327522 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327539 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327559 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327581 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327608 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327631 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327673 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327704 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmh98\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327728 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327757 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327793 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.327819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327817 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327860 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327889 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327918 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327948 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.327975 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328023 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328061 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328352 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328323 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328703 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328382 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328835 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.328964 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.328921 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.330223 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.329484 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.331266 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.330433 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.331266 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.330957 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.331441 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.331417 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.331901 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.331855 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.333703 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.334020 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334278 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.334120 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334278 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.334169 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334836 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.334805 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.334972 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.334933 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.335621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.335595 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.336121 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.336099 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.336690 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.336668 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.336894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.336875 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.338413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.338271 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmh98\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98\") pod \"prometheus-k8s-0\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.405430 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.405378 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:12:49.892129 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.892091 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:12:49.896162 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:49.896131 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468abcb6_ed71_4518_9010_d79db459f16b.slice/crio-4b5dbcdfcb8af5b81561c04d74e0f529326cb7280613658050429f94c342cb27 WatchSource:0}: Error finding container 4b5dbcdfcb8af5b81561c04d74e0f529326cb7280613658050429f94c342cb27: Status 404 returned error can't find the container with id 4b5dbcdfcb8af5b81561c04d74e0f529326cb7280613658050429f94c342cb27 Jun 12 17:12:49.927506 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:49.927477 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5df4d4978c-5t5q2" Jun 12 17:12:50.018527 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.018435 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" event={"ID":"8b28f84f-ec7e-4077-87f1-7ebb168b8e7f","Type":"ContainerStarted","Data":"90a761d01f40f7070104918155097b2c19b3dfe9e5c8e7ba832399a29da7fa73"} Jun 12 17:12:50.019676 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.019647 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"4b5dbcdfcb8af5b81561c04d74e0f529326cb7280613658050429f94c342cb27"} Jun 12 17:12:50.023046 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.023021 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"4ee116d7d664c21cef0e20d2b243886fcd1358b348cea6ea94972bf5962c0938"} Jun 12 17:12:50.023212 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.023090 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"6c5c4ffad0da5b961808d228257396af09d13a1ab0b210c0c2aa782f83b6b223"} Jun 12 17:12:50.023212 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.023138 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" event={"ID":"ecc155ca-673e-48a4-801b-dd081e8a099c","Type":"ContainerStarted","Data":"bceb17f7a9e4dcbb8e0898f4365be33e8d02e11e60b78ff39aa18a8800308e1b"} Jun 12 17:12:50.023212 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.023181 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:50.113161 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.113098 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" podStartSLOduration=2.354487049 podStartE2EDuration="6.113079321s" totalImagesPullingTime="3.758592272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:50.112347489 +0000 UTC m=+86.984995625" watchObservedRunningTime="2026-06-12 17:12:50.113079321 +0000 UTC m=+86.985727496" Jun 12 17:12:50.113367 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.113222 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" podStartSLOduration=1.427694844 podStartE2EDuration="3.113217029s" totalImagesPullingTime="1.685522185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:50.082038435 +0000 UTC m=+86.954686564" watchObservedRunningTime="2026-06-12 17:12:50.113217029 +0000 UTC m=+86.985865138" Jun 12 17:12:50.640973 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.640933 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:50.641178 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.641018 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:50.641651 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.641626 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/14df4b50-7be7-4b74-aec5-e6fd2430074f-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:50.643737 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.643715 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/14df4b50-7be7-4b74-aec5-e6fd2430074f-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5sgrm\" (UID: \"14df4b50-7be7-4b74-aec5-e6fd2430074f\") " pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:50.762757 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.762715 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5sgrm" Jun 12 17:12:50.973445 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:50.973401 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5sgrm"] Jun 12 17:12:50.991224 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:12:50.991189 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14df4b50_7be7_4b74_aec5_e6fd2430074f.slice/crio-49e000968cf9a06d87b8de7c8a3783db0c3855703c8873b264d4c987e20060b6 WatchSource:0}: Error finding container 49e000968cf9a06d87b8de7c8a3783db0c3855703c8873b264d4c987e20060b6: Status 404 returned error can't find the container with id 49e000968cf9a06d87b8de7c8a3783db0c3855703c8873b264d4c987e20060b6 Jun 12 17:12:51.027917 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:51.027879 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5sgrm" event={"ID":"14df4b50-7be7-4b74-aec5-e6fd2430074f","Type":"ContainerStarted","Data":"49e000968cf9a06d87b8de7c8a3783db0c3855703c8873b264d4c987e20060b6"} Jun 12 17:12:51.029520 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:51.029491 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" exitCode=0 Jun 12 17:12:51.029622 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:51.029579 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} Jun 12 17:12:52.540930 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:52.540879 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-64645596dd-2k82v" podUID="e34f9da6-3036-4d03-b2dd-9890114b07a4" containerName="registry" containerID="cri-o://e0365bdd40cbfc35add31b960adaa6f06c1fd8366513f11137a9339ecb9e109d" gracePeriod=30 Jun 12 17:12:54.045514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:54.045471 2544 generic.go:356] "Generic (PLEG): container finished" podID="e34f9da6-3036-4d03-b2dd-9890114b07a4" containerID="e0365bdd40cbfc35add31b960adaa6f06c1fd8366513f11137a9339ecb9e109d" exitCode=0 Jun 12 17:12:54.045968 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:54.045554 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64645596dd-2k82v" event={"ID":"e34f9da6-3036-4d03-b2dd-9890114b07a4","Type":"ContainerDied","Data":"e0365bdd40cbfc35add31b960adaa6f06c1fd8366513f11137a9339ecb9e109d"} Jun 12 17:12:55.318243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.318218 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:55.489752 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.489714 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.489947 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.489758 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.489947 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.489786 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.489947 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.489836 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.490246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490215 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.490406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490331 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.490406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490370 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k86zq\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.490406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490400 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted\") pod \"e34f9da6-3036-4d03-b2dd-9890114b07a4\" (UID: \"e34f9da6-3036-4d03-b2dd-9890114b07a4\") " Jun 12 17:12:55.490585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490571 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:12:55.490645 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490613 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:12:55.490963 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490857 2544 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-trusted-ca\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.490963 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.490885 2544 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-certificates\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.494116 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.494060 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:12:55.494240 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.494130 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:12:55.494678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.494638 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:12:55.494781 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.494753 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq" (OuterVolumeSpecName: "kube-api-access-k86zq") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "kube-api-access-k86zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:12:55.494968 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.494942 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:12:55.501972 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.501940 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "e34f9da6-3036-4d03-b2dd-9890114b07a4" (UID: "e34f9da6-3036-4d03-b2dd-9890114b07a4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591576 2544 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-registry-tls\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591612 2544 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-bound-sa-token\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591628 2544 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-image-registry-private-configuration\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591645 2544 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e34f9da6-3036-4d03-b2dd-9890114b07a4-installation-pull-secrets\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591661 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k86zq\" (UniqueName: \"kubernetes.io/projected/e34f9da6-3036-4d03-b2dd-9890114b07a4-kube-api-access-k86zq\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:55.591678 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:55.591671 2544 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e34f9da6-3036-4d03-b2dd-9890114b07a4-ca-trust-extracted\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:12:56.040407 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.040352 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-c65cbb6b-gwbb5" Jun 12 17:12:56.058157 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.058121 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} Jun 12 17:12:56.058336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.058207 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} Jun 12 17:12:56.058336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.058252 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} Jun 12 17:12:56.058336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.058321 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} Jun 12 17:12:56.058496 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.058361 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} Jun 12 17:12:56.060864 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.060831 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5sgrm" event={"ID":"14df4b50-7be7-4b74-aec5-e6fd2430074f","Type":"ContainerStarted","Data":"7a2aff24fe9769abf70c277c833d5db99b45222bff64190fed8f57156ac1d7fc"} Jun 12 17:12:56.063580 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.063554 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-64645596dd-2k82v" event={"ID":"e34f9da6-3036-4d03-b2dd-9890114b07a4","Type":"ContainerDied","Data":"a141ec2b1bc4e8d10eeb907a3330aec6108cf021a253ecb172db372588dee673"} Jun 12 17:12:56.063710 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.063630 2544 scope.go:117] "RemoveContainer" containerID="e0365bdd40cbfc35add31b960adaa6f06c1fd8366513f11137a9339ecb9e109d" Jun 12 17:12:56.063775 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.063574 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-64645596dd-2k82v" Jun 12 17:12:56.090036 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.090006 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:12:56.098235 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:56.098167 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-64645596dd-2k82v"] Jun 12 17:12:57.072553 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:57.072508 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5sgrm" event={"ID":"14df4b50-7be7-4b74-aec5-e6fd2430074f","Type":"ContainerStarted","Data":"66dd3db925e1d9552a7cc421a28ff8f23a10513ff5301bd251e36df81fe90947"} Jun 12 17:12:57.077865 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:57.077835 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerStarted","Data":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} Jun 12 17:12:57.120941 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:57.120876 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.53632589 podStartE2EDuration="8.120851998s" totalImagesPullingTime="5.559915253s" totalInitContainerRuntime="24.610855ms" isStatefulPod=false podCreationTimestamp="2026-06-12 17:12:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:57.120065527 +0000 UTC m=+93.992713638" watchObservedRunningTime="2026-06-12 17:12:57.120851998 +0000 UTC m=+93.993500109" Jun 12 17:12:57.603825 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:57.603783 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e34f9da6-3036-4d03-b2dd-9890114b07a4" path="/var/lib/kubelet/pods/e34f9da6-3036-4d03-b2dd-9890114b07a4/volumes" Jun 12 17:12:58.083693 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:58.083655 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5sgrm" event={"ID":"14df4b50-7be7-4b74-aec5-e6fd2430074f","Type":"ContainerStarted","Data":"79f84adaa44b8bac708f8112008463f84bce3844b71f29de833a0d13538a6a82"} Jun 12 17:12:58.108585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:58.108434 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5sgrm" podStartSLOduration=17.390039174 podStartE2EDuration="24.108410504s" totalImagesPullingTime="6.71837133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:12:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:12:58.107078692 +0000 UTC m=+94.979726802" watchObservedRunningTime="2026-06-12 17:12:58.108410504 +0000 UTC m=+94.981058614" Jun 12 17:12:59.406277 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:12:59.406233 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:13:07.572580 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:07.572536 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:13:07.572977 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:07.572846 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:13:07.962307 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:07.962256 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-bcj25" Jun 12 17:13:27.577983 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:27.577951 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:13:27.581919 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:27.581890 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5b5ff785fc-jmv2h" Jun 12 17:13:49.406406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:49.406355 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:13:49.423452 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:49.423423 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:13:50.288216 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:13:50.288186 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:07.611861 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.611825 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612221 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="prometheus" containerID="cri-o://b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" gracePeriod=600 Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612250 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy" containerID="cri-o://1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" gracePeriod=600 Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612264 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="thanos-sidecar" containerID="cri-o://f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" gracePeriod=600 Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612327 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" gracePeriod=600 Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612332 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="config-reloader" containerID="cri-o://dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" gracePeriod=600 Jun 12 17:14:07.612480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.612310 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-web" containerID="cri-o://6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" gracePeriod=600 Jun 12 17:14:07.859545 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.859520 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:07.881506 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881425 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881506 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881465 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881506 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881483 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881506 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881505 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gmh98\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881528 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881553 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881583 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881605 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881639 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881699 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881727 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.881788 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881767 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881796 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881820 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881861 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881892 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881887 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881917 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881920 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881948 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.881989 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882182 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.882023 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config\") pod \"468abcb6-ed71-4518-9010-d79db459f16b\" (UID: \"468abcb6-ed71-4518-9010-d79db459f16b\") " Jun 12 17:14:07.882683 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.882459 2544 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-metrics-client-ca\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.882683 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.882480 2544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.883064 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.883034 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.884339 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.883381 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.884339 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.883510 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.884339 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.884174 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.884989 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.884778 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:14:07.885791 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.885756 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.885895 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.885833 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.885895 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.885838 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.886767 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.886738 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:14:07.887349 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887262 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out" (OuterVolumeSpecName: "config-out") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:14:07.887349 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887261 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.887349 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887283 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.887493 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887351 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.887493 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887370 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98" (OuterVolumeSpecName: "kube-api-access-gmh98") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "kube-api-access-gmh98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:14:07.887493 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887442 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config" (OuterVolumeSpecName: "config") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.887845 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887825 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:14:07.887894 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.887878 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.898872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.898840 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config" (OuterVolumeSpecName: "web-config") pod "468abcb6-ed71-4518-9010-d79db459f16b" (UID: "468abcb6-ed71-4518-9010-d79db459f16b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:14:07.983019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.982980 2544 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-config\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983016 2544 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983027 2544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983038 2544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983048 2544 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-config-out\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983059 2544 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983068 2544 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-metrics-client-certs\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983076 2544 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983084 2544 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-tls-assets\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983092 2544 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-grpc-tls\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983100 2544 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-web-config\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983110 2544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-tls\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983118 2544 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-thanos-prometheus-http-client-file\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983128 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gmh98\" (UniqueName: \"kubernetes.io/projected/468abcb6-ed71-4518-9010-d79db459f16b-kube-api-access-gmh98\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983136 2544 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983145 2544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-db\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983153 2544 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/468abcb6-ed71-4518-9010-d79db459f16b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:07.983246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:07.983161 2544 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/468abcb6-ed71-4518-9010-d79db459f16b-secret-kube-rbac-proxy\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340427 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340456 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340467 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340477 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340485 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340491 2544 generic.go:356] "Generic (PLEG): container finished" podID="468abcb6-ed71-4518-9010-d79db459f16b" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" exitCode=0 Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340507 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.340521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340509 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340580 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340608 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340633 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340652 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340663 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340689 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} Jun 12 17:14:08.340946 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.340713 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"468abcb6-ed71-4518-9010-d79db459f16b","Type":"ContainerDied","Data":"4b5dbcdfcb8af5b81561c04d74e0f529326cb7280613658050429f94c342cb27"} Jun 12 17:14:08.348576 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.348558 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.356090 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.356071 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.363399 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.363377 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.368577 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.368512 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:08.371794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.371775 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.377978 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.377953 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:08.379621 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.379602 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.387415 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.387396 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.394437 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.394417 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.394741 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.394721 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.394809 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.394751 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.394809 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.394785 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.395036 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.395013 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.395088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395045 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.395088 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395063 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.395330 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.395314 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.395389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395335 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.395389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395348 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.395594 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.395578 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.395633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395599 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.395633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395613 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.395825 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.395809 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.395865 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395829 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.395865 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.395840 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.396044 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.396029 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.396092 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396048 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.396092 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396059 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.396261 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:14:08.396243 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.396320 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396266 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.396320 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396278 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.396498 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396482 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.396542 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396498 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.396706 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396689 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.396759 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396705 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.396945 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396922 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.396945 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.396941 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.397154 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397139 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.397192 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397155 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.397399 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397384 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.397443 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397400 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.397585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397569 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.397642 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397586 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.397792 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397775 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.397829 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397793 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.397988 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397973 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.398030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.397988 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.398186 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398168 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.398227 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398187 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.398398 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398381 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.398464 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398398 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.398634 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398618 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.398634 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398634 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.398845 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398828 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.398896 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.398846 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.399075 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399060 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.399075 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399075 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.399276 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399259 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.399342 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399279 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.399514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399497 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.399560 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399514 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.399728 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399711 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.399784 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399727 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.399909 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399891 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.399949 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.399911 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.400114 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400078 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.400153 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400115 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.400284 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400266 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.400360 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400285 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.400521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400503 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.400569 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400520 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.400736 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400720 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.400787 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400736 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.400943 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400926 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.400991 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.400943 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.401155 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401139 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.401196 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401154 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.401386 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401370 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.401437 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401387 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.401591 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401573 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.401591 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401590 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.401789 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401771 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.401826 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.401789 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.402021 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402005 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.402068 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402021 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.402217 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402199 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.402258 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402219 2544 scope.go:117] "RemoveContainer" containerID="758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679" Jun 12 17:14:08.402489 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402471 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679"} err="failed to get container status \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": rpc error: code = NotFound desc = could not find container \"758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679\": container with ID starting with 758798b72865f06e2dd260c36061c88fc7b7fa2dad4e9357f9bdd625a8873679 not found: ID does not exist" Jun 12 17:14:08.402549 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402490 2544 scope.go:117] "RemoveContainer" containerID="1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018" Jun 12 17:14:08.402705 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402682 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018"} err="failed to get container status \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": rpc error: code = NotFound desc = could not find container \"1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018\": container with ID starting with 1cb1dd34a9cfaf309aa4385c5b48088e1af5b97b5f23997ac1f8b7ebbc958018 not found: ID does not exist" Jun 12 17:14:08.402750 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402706 2544 scope.go:117] "RemoveContainer" containerID="6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843" Jun 12 17:14:08.402891 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402875 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843"} err="failed to get container status \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": rpc error: code = NotFound desc = could not find container \"6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843\": container with ID starting with 6826cb9e566d5cc360621b036ae446299705923def03e60368df7573193bb843 not found: ID does not exist" Jun 12 17:14:08.402929 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.402892 2544 scope.go:117] "RemoveContainer" containerID="f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed" Jun 12 17:14:08.403122 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403107 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed"} err="failed to get container status \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": rpc error: code = NotFound desc = could not find container \"f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed\": container with ID starting with f3ad1a0a3e4c0999fc0f50f600c71512b505b9cb04d99b233b0118b292c747ed not found: ID does not exist" Jun 12 17:14:08.403160 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403122 2544 scope.go:117] "RemoveContainer" containerID="dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368" Jun 12 17:14:08.403336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403320 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368"} err="failed to get container status \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": rpc error: code = NotFound desc = could not find container \"dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368\": container with ID starting with dc50529a0508119ff3e79643d011789b926d6b89553151c395dbe22c3676f368 not found: ID does not exist" Jun 12 17:14:08.403384 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403336 2544 scope.go:117] "RemoveContainer" containerID="b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43" Jun 12 17:14:08.403617 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403602 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43"} err="failed to get container status \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": rpc error: code = NotFound desc = could not find container \"b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43\": container with ID starting with b7398db35e5f022300097e6b6c838baa1a9dc73070c813d3fbe2a1acfeceda43 not found: ID does not exist" Jun 12 17:14:08.403658 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403618 2544 scope.go:117] "RemoveContainer" containerID="e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76" Jun 12 17:14:08.403835 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.403819 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76"} err="failed to get container status \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": rpc error: code = NotFound desc = could not find container \"e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76\": container with ID starting with e61b7a47a177c82f73260111503f385ed37d86cc210ea48804b885a4fb972f76 not found: ID does not exist" Jun 12 17:14:08.442450 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442413 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:08.442674 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442662 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e34f9da6-3036-4d03-b2dd-9890114b07a4" containerName="registry" Jun 12 17:14:08.442717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442676 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="e34f9da6-3036-4d03-b2dd-9890114b07a4" containerName="registry" Jun 12 17:14:08.442717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442685 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="init-config-reloader" Jun 12 17:14:08.442717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442691 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="init-config-reloader" Jun 12 17:14:08.442717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442698 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="prometheus" Jun 12 17:14:08.442717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442703 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="prometheus" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442722 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="config-reloader" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442728 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="config-reloader" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442733 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442738 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442744 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-thanos" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442749 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-thanos" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442755 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="thanos-sidecar" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442760 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="thanos-sidecar" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442766 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-web" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442771 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-web" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442808 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="config-reloader" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442814 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="prometheus" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442821 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-web" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442827 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442833 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="thanos-sidecar" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442838 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="468abcb6-ed71-4518-9010-d79db459f16b" containerName="kube-rbac-proxy-thanos" Jun 12 17:14:08.442862 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.442845 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="e34f9da6-3036-4d03-b2dd-9890114b07a4" containerName="registry" Jun 12 17:14:08.444914 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.444897 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.447688 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.447663 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jun 12 17:14:08.448167 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.448149 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jun 12 17:14:08.448237 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.448176 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jun 12 17:14:08.450180 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.450159 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jun 12 17:14:08.450307 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.450263 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jun 12 17:14:08.450521 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.450431 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jun 12 17:14:08.450606 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.450473 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jun 12 17:14:08.450693 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.450674 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4497pcg7og20c\"" Jun 12 17:14:08.451249 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451232 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jun 12 17:14:08.451389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451366 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jun 12 17:14:08.451467 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451454 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jun 12 17:14:08.451514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451485 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jun 12 17:14:08.451514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451485 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jun 12 17:14:08.451662 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.451650 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-rb85v\"" Jun 12 17:14:08.457809 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.457791 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jun 12 17:14:08.464681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.462671 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jun 12 17:14:08.477875 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.477846 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:08.488279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488248 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488283 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488373 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488416 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488446 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-web-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488475 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488505 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488495 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488511 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488534 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488555 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488630 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488676 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.488723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488708 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488737 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488763 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488786 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlcq5\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-kube-api-access-zlcq5\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488822 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488889 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-config-out\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488928 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.489000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.488955 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.589780 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589742 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.589780 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589788 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-config-out\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589809 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589826 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589846 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589862 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589886 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589910 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589935 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-web-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589965 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.589993 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590014 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590036 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590055 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590081 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590101 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590129 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590158 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590183 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590207 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlcq5\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-kube-api-access-zlcq5\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.590635 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590282 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.591170 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590728 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.591170 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.590899 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.591581 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.591553 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.592220 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.592194 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593427 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593179 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/a881a870-c24a-4f10-a2ac-cdd15960c697-config-out\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593427 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593189 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593427 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593258 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593613 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593513 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593697 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593677 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.593745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.593687 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.594048 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.594027 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.594127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.594103 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.594554 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.594529 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.595473 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.595446 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-web-config\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.595778 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.595757 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.596067 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.596047 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.596255 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.596235 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/a881a870-c24a-4f10-a2ac-cdd15960c697-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.597069 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.597050 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/a881a870-c24a-4f10-a2ac-cdd15960c697-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.603332 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.603310 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlcq5\" (UniqueName: \"kubernetes.io/projected/a881a870-c24a-4f10-a2ac-cdd15960c697-kube-api-access-zlcq5\") pod \"prometheus-k8s-0\" (UID: \"a881a870-c24a-4f10-a2ac-cdd15960c697\") " pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.756265 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.756209 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:14:08.907746 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:08.907713 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jun 12 17:14:08.911026 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:14:08.910999 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda881a870_c24a_4f10_a2ac_cdd15960c697.slice/crio-c05780af34ec7e5287a2a2cfccb0e4896cd9e6d0d12af212968fabf77a7daa62 WatchSource:0}: Error finding container c05780af34ec7e5287a2a2cfccb0e4896cd9e6d0d12af212968fabf77a7daa62: Status 404 returned error can't find the container with id c05780af34ec7e5287a2a2cfccb0e4896cd9e6d0d12af212968fabf77a7daa62 Jun 12 17:14:09.345762 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:09.345727 2544 generic.go:356] "Generic (PLEG): container finished" podID="a881a870-c24a-4f10-a2ac-cdd15960c697" containerID="f085c1bb3d41a1efa4ea4e3f78849aad1fecb6e523b09d924bfabdff8e6f9e35" exitCode=0 Jun 12 17:14:09.345949 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:09.345843 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerDied","Data":"f085c1bb3d41a1efa4ea4e3f78849aad1fecb6e523b09d924bfabdff8e6f9e35"} Jun 12 17:14:09.345949 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:09.345918 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"c05780af34ec7e5287a2a2cfccb0e4896cd9e6d0d12af212968fabf77a7daa62"} Jun 12 17:14:09.604980 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:09.604949 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468abcb6-ed71-4518-9010-d79db459f16b" path="/var/lib/kubelet/pods/468abcb6-ed71-4518-9010-d79db459f16b/volumes" Jun 12 17:14:10.354595 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354509 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"61aa3ca1af53a0fecba35d68ae89e66e9dece40afbcc91e39bb729400dafdb41"} Jun 12 17:14:10.354595 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354585 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"f745be518e9817aefb5ffda88f8737a0a3d5d277ed7166fb426a86bddfe0c468"} Jun 12 17:14:10.355080 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354633 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"5a34eb182f87c3b36c3a3da00bb1959917889126a5597f0549c8aa0692e8360c"} Jun 12 17:14:10.355080 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354671 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"88bdf2728254e5c0af413ca3adb3bdd339a70a47310c0cb93a7b4d45fd0da9c3"} Jun 12 17:14:10.355080 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354710 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"775fb1a8e4d1b57f92b41977aa5bac85b4651eb8b12c4af193eb2863bfc94f17"} Jun 12 17:14:10.355080 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.354754 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"a881a870-c24a-4f10-a2ac-cdd15960c697","Type":"ContainerStarted","Data":"db2ca21853dc05ace50a4e00160d52c6cdc4c903892834df965cc7b62b91cb32"} Jun 12 17:14:10.394035 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:10.393963 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.373382403 podStartE2EDuration="2.393938374s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.555971ms" isStatefulPod=false podCreationTimestamp="2026-06-12 17:14:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:14:10.392038581 +0000 UTC m=+167.264686691" watchObservedRunningTime="2026-06-12 17:14:10.393938374 +0000 UTC m=+167.266586674" Jun 12 17:14:13.756821 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:14:13.756768 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:15:08.757082 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:15:08.757040 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:15:08.773936 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:15:08.773906 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:15:09.588964 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:15:09.588932 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jun 12 17:16:23.544177 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:16:23.544147 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:16:23.545335 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:16:23.545289 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:16:23.547013 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:16:23.546991 2544 kubelet.go:1648] "Image garbage collection succeeded" Jun 12 17:17:13.323763 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.323680 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5"] Jun 12 17:17:13.325830 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.325812 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.328100 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.328075 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 12 17:17:13.328540 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.328523 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 12 17:17:13.328654 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.328524 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-m99kb\"" Jun 12 17:17:13.334480 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.334456 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5"] Jun 12 17:17:13.377814 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.377765 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwcqw\" (UniqueName: \"kubernetes.io/projected/95df3d51-8124-44b2-9cf2-fe6f7c773261-kube-api-access-wwcqw\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.377814 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.377812 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95df3d51-8124-44b2-9cf2-fe6f7c773261-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.478717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.478679 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwcqw\" (UniqueName: \"kubernetes.io/projected/95df3d51-8124-44b2-9cf2-fe6f7c773261-kube-api-access-wwcqw\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.478717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.478724 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95df3d51-8124-44b2-9cf2-fe6f7c773261-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.479135 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.479113 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95df3d51-8124-44b2-9cf2-fe6f7c773261-tmp\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.487052 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.487027 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwcqw\" (UniqueName: \"kubernetes.io/projected/95df3d51-8124-44b2-9cf2-fe6f7c773261-kube-api-access-wwcqw\") pod \"openshift-lws-operator-fd8ccff4c-2xjp5\" (UID: \"95df3d51-8124-44b2-9cf2-fe6f7c773261\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.646872 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.646833 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" Jun 12 17:17:13.787542 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.787500 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5"] Jun 12 17:17:13.792164 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:17:13.792125 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95df3d51_8124_44b2_9cf2_fe6f7c773261.slice/crio-0cf8249832ec6e538ec57ddc4482506b4acc814c81a4a47cf6a62f3f8c00eb11 WatchSource:0}: Error finding container 0cf8249832ec6e538ec57ddc4482506b4acc814c81a4a47cf6a62f3f8c00eb11: Status 404 returned error can't find the container with id 0cf8249832ec6e538ec57ddc4482506b4acc814c81a4a47cf6a62f3f8c00eb11 Jun 12 17:17:13.793725 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:13.793705 2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 17:17:14.026551 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:14.026458 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" event={"ID":"95df3d51-8124-44b2-9cf2-fe6f7c773261","Type":"ContainerStarted","Data":"0cf8249832ec6e538ec57ddc4482506b4acc814c81a4a47cf6a62f3f8c00eb11"} Jun 12 17:17:17.042173 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:17.042140 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" event={"ID":"95df3d51-8124-44b2-9cf2-fe6f7c773261","Type":"ContainerStarted","Data":"d842fa06ae3b57bb47549075861c76777ccc20140d3c62677640eb4ee23e2a36"} Jun 12 17:17:17.069020 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:17.068961 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-2xjp5" podStartSLOduration=1.264624967 podStartE2EDuration="4.0689448s" totalImagesPullingTime="2.804319833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:17:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:17:17.067390798 +0000 UTC m=+353.940038906" watchObservedRunningTime="2026-06-12 17:17:17.0689448 +0000 UTC m=+353.941592911" Jun 12 17:17:33.205534 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.205494 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc"] Jun 12 17:17:33.209018 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.208999 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.211108 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.211082 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-n4h4s\"" Jun 12 17:17:33.211249 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.211204 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 12 17:17:33.211313 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.211273 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 12 17:17:33.211462 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.211449 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 12 17:17:33.211893 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.211873 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 12 17:17:33.226823 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.226790 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc"] Jun 12 17:17:33.251103 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.251071 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk5nv\" (UniqueName: \"kubernetes.io/projected/cfaa229a-3910-42ed-946d-a6b5c89fad58-kube-api-access-zk5nv\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.251103 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.251107 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-tmp\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.251328 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.251131 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-manifests\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.251328 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.251155 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.251328 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.251196 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352238 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-tmp\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352280 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-manifests\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352566 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352342 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352566 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352373 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352566 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352475 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk5nv\" (UniqueName: \"kubernetes.io/projected/cfaa229a-3910-42ed-946d-a6b5c89fad58-kube-api-access-zk5nv\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.352790 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.352755 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-manifests\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.354745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.354719 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/cfaa229a-3910-42ed-946d-a6b5c89fad58-tmp\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.355442 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.355420 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-apiservice-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.355565 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.355546 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfaa229a-3910-42ed-946d-a6b5c89fad58-webhook-cert\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.361216 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.361188 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk5nv\" (UniqueName: \"kubernetes.io/projected/cfaa229a-3910-42ed-946d-a6b5c89fad58-kube-api-access-zk5nv\") pod \"opendatahub-operator-controller-manager-d86876fc-bpbwc\" (UID: \"cfaa229a-3910-42ed-946d-a6b5c89fad58\") " pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.520314 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.520185 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:33.679360 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:33.679328 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc"] Jun 12 17:17:33.683204 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:17:33.683160 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfaa229a_3910_42ed_946d_a6b5c89fad58.slice/crio-8c0607ffbf55aff713df3552946741b95b24f6e2b8e5687fada4737b469768d2 WatchSource:0}: Error finding container 8c0607ffbf55aff713df3552946741b95b24f6e2b8e5687fada4737b469768d2: Status 404 returned error can't find the container with id 8c0607ffbf55aff713df3552946741b95b24f6e2b8e5687fada4737b469768d2 Jun 12 17:17:34.108378 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:34.108330 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" event={"ID":"cfaa229a-3910-42ed-946d-a6b5c89fad58","Type":"ContainerStarted","Data":"8c0607ffbf55aff713df3552946741b95b24f6e2b8e5687fada4737b469768d2"} Jun 12 17:17:37.123066 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:37.123029 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" event={"ID":"cfaa229a-3910-42ed-946d-a6b5c89fad58","Type":"ContainerStarted","Data":"9f654b6b71b32d4fdfb57a31052919485adf32cae20a1862b9336dea1c6a32b3"} Jun 12 17:17:38.128818 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:38.128778 2544 generic.go:356] "Generic (PLEG): container finished" podID="cfaa229a-3910-42ed-946d-a6b5c89fad58" containerID="9f654b6b71b32d4fdfb57a31052919485adf32cae20a1862b9336dea1c6a32b3" exitCode=0 Jun 12 17:17:38.129194 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:38.128876 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" event={"ID":"cfaa229a-3910-42ed-946d-a6b5c89fad58","Type":"ContainerDied","Data":"9f654b6b71b32d4fdfb57a31052919485adf32cae20a1862b9336dea1c6a32b3"} Jun 12 17:17:39.135119 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:39.135085 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" event={"ID":"cfaa229a-3910-42ed-946d-a6b5c89fad58","Type":"ContainerStarted","Data":"f6ecab0b2e4a1048e88c5aa81a228f2648adea96ad68baa698b2abc7df5fa8a6"} Jun 12 17:17:39.135517 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:39.135221 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:39.155955 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:39.155899 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" podStartSLOduration=1.665745109 podStartE2EDuration="6.155883782s" totalImagesPullingTime="3.515011484s" totalInitContainerRuntime="975.127189ms" isStatefulPod=false podCreationTimestamp="2026-06-12 17:17:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:17:39.154335305 +0000 UTC m=+376.026983437" watchObservedRunningTime="2026-06-12 17:17:39.155883782 +0000 UTC m=+376.028531958" Jun 12 17:17:50.141500 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:50.141471 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-d86876fc-bpbwc" Jun 12 17:17:51.193769 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.193730 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4"] Jun 12 17:17:51.293944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.293905 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4"] Jun 12 17:17:51.294116 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.294035 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.296519 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.296500 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-h4v8d\"" Jun 12 17:17:51.296746 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.296731 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 12 17:17:51.297107 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.297092 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 12 17:17:51.297253 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.297238 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 12 17:17:51.404453 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.404409 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.404630 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.404522 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3a0cde-4422-4306-9105-223d6499ed18-manager-config\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.404630 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.404555 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-metrics-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.404706 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.404625 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fm6cb\" (UniqueName: \"kubernetes.io/projected/7a3a0cde-4422-4306-9105-223d6499ed18-kube-api-access-fm6cb\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.505194 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.505095 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fm6cb\" (UniqueName: \"kubernetes.io/projected/7a3a0cde-4422-4306-9105-223d6499ed18-kube-api-access-fm6cb\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.505194 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.505138 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.505194 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.505193 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3a0cde-4422-4306-9105-223d6499ed18-manager-config\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.505517 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.505214 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-metrics-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.505899 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.505879 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7a3a0cde-4422-4306-9105-223d6499ed18-manager-config\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.507922 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.507897 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-metrics-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.508015 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.507944 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a3a0cde-4422-4306-9105-223d6499ed18-cert\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.516073 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.516047 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fm6cb\" (UniqueName: \"kubernetes.io/projected/7a3a0cde-4422-4306-9105-223d6499ed18-kube-api-access-fm6cb\") pod \"lws-controller-manager-574bbc8b44-lb7k4\" (UID: \"7a3a0cde-4422-4306-9105-223d6499ed18\") " pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.604473 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.604434 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:51.874161 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:17:51.874133 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3a0cde_4422_4306_9105_223d6499ed18.slice/crio-d7c2f229fa65282d87d77aeeda892424919b18ec7e27ea128546f39fa66e3857 WatchSource:0}: Error finding container d7c2f229fa65282d87d77aeeda892424919b18ec7e27ea128546f39fa66e3857: Status 404 returned error can't find the container with id d7c2f229fa65282d87d77aeeda892424919b18ec7e27ea128546f39fa66e3857 Jun 12 17:17:51.888245 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:51.888212 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4"] Jun 12 17:17:52.187859 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:52.187776 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" event={"ID":"7a3a0cde-4422-4306-9105-223d6499ed18","Type":"ContainerStarted","Data":"d7c2f229fa65282d87d77aeeda892424919b18ec7e27ea128546f39fa66e3857"} Jun 12 17:17:54.197674 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:54.197634 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" event={"ID":"7a3a0cde-4422-4306-9105-223d6499ed18","Type":"ContainerStarted","Data":"c9fe60ce82271e16daef4e6f229b648f745ae41f09196d014cd3ccbe5eaf146b"} Jun 12 17:17:54.198225 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:54.197847 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:17:54.222782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:17:54.222721 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" podStartSLOduration=1.6555378840000001 podStartE2EDuration="3.222698701s" totalImagesPullingTime="1.567160817s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:17:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:17:54.22188192 +0000 UTC m=+391.094530030" watchObservedRunningTime="2026-06-12 17:17:54.222698701 +0000 UTC m=+391.095346813" Jun 12 17:18:05.205216 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:05.205180 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-574bbc8b44-lb7k4" Jun 12 17:18:20.671957 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.671925 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd"] Jun 12 17:18:20.675401 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.675370 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.683810 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:20.683752 2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"dashboard-redirect-7cbdbb8cc9-bnktd\" is forbidden: User \"system:node:ip-10-0-128-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-121.ec2.internal' and this object" podUID="ea2a518d-f699-4fc0-ab32-460eff046354" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.684116 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:20.684079 2544 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dashboard-redirect-config\" is forbidden: User \"system:node:ip-10-0-128-121.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-121.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" type="*v1.ConfigMap" Jun 12 17:18:20.684219 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:20.684176 2544 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"default-dockercfg-xcj5b\" is forbidden: User \"system:node:ip-10-0-128-121.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-121.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"opendatahub\"/\"default-dockercfg-xcj5b\"" type="*v1.Secret" Jun 12 17:18:20.707925 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.707883 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd"] Jun 12 17:18:20.745131 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.745072 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhwf\" (UniqueName: \"kubernetes.io/projected/ea2a518d-f699-4fc0-ab32-460eff046354-kube-api-access-mdhwf\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.745349 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.745197 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.846131 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.846083 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhwf\" (UniqueName: \"kubernetes.io/projected/ea2a518d-f699-4fc0-ab32-460eff046354-kube-api-access-mdhwf\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.846354 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.846175 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:20.881415 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:20.881383 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhwf\" (UniqueName: \"kubernetes.io/projected/ea2a518d-f699-4fc0-ab32-460eff046354-kube-api-access-mdhwf\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:21.846565 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:21.846521 2544 configmap.go:193] Couldn't get configMap opendatahub/dashboard-redirect-config: failed to sync configmap cache: timed out waiting for the condition Jun 12 17:18:21.846976 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:21.846599 2544 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config podName:ea2a518d-f699-4fc0-ab32-460eff046354 nodeName:}" failed. No retries permitted until 2026-06-12 17:18:22.346581462 +0000 UTC m=+419.219229549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "redirect-config" (UniqueName: "kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config") pod "dashboard-redirect-7cbdbb8cc9-bnktd" (UID: "ea2a518d-f699-4fc0-ab32-460eff046354") : failed to sync configmap cache: timed out waiting for the condition Jun 12 17:18:21.980804 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:21.980774 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xcj5b\"" Jun 12 17:18:22.078759 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:22.078725 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"dashboard-redirect-config\"" Jun 12 17:18:22.360164 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:22.360121 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:22.360803 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:22.360784 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"redirect-config\" (UniqueName: \"kubernetes.io/configmap/ea2a518d-f699-4fc0-ab32-460eff046354-redirect-config\") pod \"dashboard-redirect-7cbdbb8cc9-bnktd\" (UID: \"ea2a518d-f699-4fc0-ab32-460eff046354\") " pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:22.487457 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:22.487420 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" Jun 12 17:18:22.630555 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:22.630476 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd"] Jun 12 17:18:22.633860 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:18:22.633834 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2a518d_f699_4fc0_ab32_460eff046354.slice/crio-8d02142ebd41aa6ba276c1b16f015fef84369de4c3da5218fab9b37276d1307b WatchSource:0}: Error finding container 8d02142ebd41aa6ba276c1b16f015fef84369de4c3da5218fab9b37276d1307b: Status 404 returned error can't find the container with id 8d02142ebd41aa6ba276c1b16f015fef84369de4c3da5218fab9b37276d1307b Jun 12 17:18:23.316002 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:23.315961 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" event={"ID":"ea2a518d-f699-4fc0-ab32-460eff046354","Type":"ContainerStarted","Data":"8d02142ebd41aa6ba276c1b16f015fef84369de4c3da5218fab9b37276d1307b"} Jun 12 17:18:25.327503 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:25.327462 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" event={"ID":"ea2a518d-f699-4fc0-ab32-460eff046354","Type":"ContainerStarted","Data":"6d80105b0100d8d7717f13b4e461bdfe2a27492081275848919a63d1c46fde02"} Jun 12 17:18:25.347626 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:25.347568 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/dashboard-redirect-7cbdbb8cc9-bnktd" podStartSLOduration=3.312875604 podStartE2EDuration="5.347552726s" totalImagesPullingTime="2.034677122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:18:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:18:25.346605778 +0000 UTC m=+422.219253908" watchObservedRunningTime="2026-06-12 17:18:25.347552726 +0000 UTC m=+422.220200835" Jun 12 17:18:34.819409 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.819367 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t"] Jun 12 17:18:34.824925 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.824900 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.828344 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.828319 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xjf77\"" Jun 12 17:18:34.828731 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.828712 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jun 12 17:18:34.841457 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.841416 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t"] Jun 12 17:18:34.877737 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877692 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877737 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877731 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877754 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a13ca2-c060-42f8-bd5c-466ae88f3508-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877773 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877791 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877872 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.877999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.877903 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5pxt\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-kube-api-access-q5pxt\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.878243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.878016 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.878243 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.878047 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979042 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.978995 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979042 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979039 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979334 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979174 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979334 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979210 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979334 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979232 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a13ca2-c060-42f8-bd5c-466ae88f3508-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979334 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979254 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979552 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979367 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979552 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979428 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979552 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979455 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5pxt\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-kube-api-access-q5pxt\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979552 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979537 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979769 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-data\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.979812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979804 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.980170 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.979874 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.980170 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.980032 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/22a13ca2-c060-42f8-bd5c-466ae88f3508-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.981809 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.981787 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.982080 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.982066 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.991151 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.991116 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5pxt\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-kube-api-access-q5pxt\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:34.991317 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:34.991122 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/22a13ca2-c060-42f8-bd5c-466ae88f3508-istio-token\") pod \"data-science-gateway-data-science-gateway-class-5b556588bcjh74t\" (UID: \"22a13ca2-c060-42f8-bd5c-466ae88f3508\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:35.136365 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:35.136327 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:35.287679 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:35.287576 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t"] Jun 12 17:18:35.289961 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:18:35.289934 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22a13ca2_c060_42f8_bd5c_466ae88f3508.slice/crio-e65047743ab98e087e4d5bcb6c894a1a646358a09bd320a7c70fe28b5f56cf56 WatchSource:0}: Error finding container e65047743ab98e087e4d5bcb6c894a1a646358a09bd320a7c70fe28b5f56cf56: Status 404 returned error can't find the container with id e65047743ab98e087e4d5bcb6c894a1a646358a09bd320a7c70fe28b5f56cf56 Jun 12 17:18:35.369432 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:35.369393 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" event={"ID":"22a13ca2-c060-42f8-bd5c-466ae88f3508","Type":"ContainerStarted","Data":"e65047743ab98e087e4d5bcb6c894a1a646358a09bd320a7c70fe28b5f56cf56"} Jun 12 17:18:38.629077 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:38.629038 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:18:38.629420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:38.629122 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:18:38.629420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:38.629155 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:18:39.390791 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:39.390754 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" event={"ID":"22a13ca2-c060-42f8-bd5c-466ae88f3508","Type":"ContainerStarted","Data":"ebb3fe8bd8dd77b2788908f688a87809b9592b45cf555b4ae9e5e32f09894942"} Jun 12 17:18:39.414093 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:39.414034 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" podStartSLOduration=2.077221542 podStartE2EDuration="5.414012049s" totalImagesPullingTime="3.336790507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:18:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:18:39.413632339 +0000 UTC m=+436.286280447" watchObservedRunningTime="2026-06-12 17:18:39.414012049 +0000 UTC m=+436.286660159" Jun 12 17:18:40.137389 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:40.137276 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:40.142232 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:40.142205 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:40.394819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:40.394723 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:40.395728 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:40.395706 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-5b556588bcjh74t" Jun 12 17:18:53.737501 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.737461 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:53.740882 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.740851 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:53.743207 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.743185 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-k5h8c\"" Jun 12 17:18:53.743986 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.743969 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 12 17:18:53.745092 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.745069 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 12 17:18:53.753680 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.753651 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:53.848038 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.847988 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24qmj\" (UniqueName: \"kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj\") pod \"kuadrant-operator-catalog-55g46\" (UID: \"94355930-b4ed-49b1-b4d3-5485efde8d36\") " pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:53.949461 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.949424 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24qmj\" (UniqueName: \"kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj\") pod \"kuadrant-operator-catalog-55g46\" (UID: \"94355930-b4ed-49b1-b4d3-5485efde8d36\") " pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:53.957818 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:53.957791 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-24qmj\" (UniqueName: \"kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj\") pod \"kuadrant-operator-catalog-55g46\" (UID: \"94355930-b4ed-49b1-b4d3-5485efde8d36\") " pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:54.051106 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.051022 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:54.059110 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.059078 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:54.195104 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.195067 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:54.198184 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:18:54.198155 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94355930_b4ed_49b1_b4d3_5485efde8d36.slice/crio-6383ad81326ed95fc0f1d61a18e50cc317cac3959becf211cb7bd1a0326a7ba2 WatchSource:0}: Error finding container 6383ad81326ed95fc0f1d61a18e50cc317cac3959becf211cb7bd1a0326a7ba2: Status 404 returned error can't find the container with id 6383ad81326ed95fc0f1d61a18e50cc317cac3959becf211cb7bd1a0326a7ba2 Jun 12 17:18:54.270758 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.270721 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-82dj6"] Jun 12 17:18:54.276760 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.276735 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:18:54.282030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.282000 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-82dj6"] Jun 12 17:18:54.453853 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.453816 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ztwt\" (UniqueName: \"kubernetes.io/projected/29fb7620-3600-40c3-91ca-03d6e371d405-kube-api-access-7ztwt\") pod \"kuadrant-operator-catalog-82dj6\" (UID: \"29fb7620-3600-40c3-91ca-03d6e371d405\") " pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:18:54.455238 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.455202 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-55g46" event={"ID":"94355930-b4ed-49b1-b4d3-5485efde8d36","Type":"ContainerStarted","Data":"6383ad81326ed95fc0f1d61a18e50cc317cac3959becf211cb7bd1a0326a7ba2"} Jun 12 17:18:54.554959 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.554921 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ztwt\" (UniqueName: \"kubernetes.io/projected/29fb7620-3600-40c3-91ca-03d6e371d405-kube-api-access-7ztwt\") pod \"kuadrant-operator-catalog-82dj6\" (UID: \"29fb7620-3600-40c3-91ca-03d6e371d405\") " pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:18:54.566559 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.566521 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ztwt\" (UniqueName: \"kubernetes.io/projected/29fb7620-3600-40c3-91ca-03d6e371d405-kube-api-access-7ztwt\") pod \"kuadrant-operator-catalog-82dj6\" (UID: \"29fb7620-3600-40c3-91ca-03d6e371d405\") " pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:18:54.587812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.587775 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:18:54.731362 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:54.731267 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-82dj6"] Jun 12 17:18:54.733438 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:18:54.733412 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fb7620_3600_40c3_91ca_03d6e371d405.slice/crio-e3e60d6109b0746ca327ff5819037f03ea53dca66897638872a828d88c8ecdc7 WatchSource:0}: Error finding container e3e60d6109b0746ca327ff5819037f03ea53dca66897638872a828d88c8ecdc7: Status 404 returned error can't find the container with id e3e60d6109b0746ca327ff5819037f03ea53dca66897638872a828d88c8ecdc7 Jun 12 17:18:55.463680 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:55.463648 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" event={"ID":"29fb7620-3600-40c3-91ca-03d6e371d405","Type":"ContainerStarted","Data":"e3e60d6109b0746ca327ff5819037f03ea53dca66897638872a828d88c8ecdc7"} Jun 12 17:18:57.473864 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.473821 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-55g46" event={"ID":"94355930-b4ed-49b1-b4d3-5485efde8d36","Type":"ContainerStarted","Data":"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76"} Jun 12 17:18:57.474424 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.473901 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-55g46" podUID="94355930-b4ed-49b1-b4d3-5485efde8d36" containerName="registry-server" containerID="cri-o://d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76" gracePeriod=2 Jun 12 17:18:57.475486 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.475458 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" event={"ID":"29fb7620-3600-40c3-91ca-03d6e371d405","Type":"ContainerStarted","Data":"6c73d5355fa1caa958f751d3458dba9e864eb0d93ca1ef3ccda98e5cb5dede77"} Jun 12 17:18:57.489096 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.489051 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-55g46" podStartSLOduration=2.235173286 podStartE2EDuration="4.489035431s" totalImagesPullingTime="2.253862145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:18:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:18:57.488325236 +0000 UTC m=+454.360973342" watchObservedRunningTime="2026-06-12 17:18:57.489035431 +0000 UTC m=+454.361683539" Jun 12 17:18:57.503120 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.503070 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" podStartSLOduration=1.781473696 podStartE2EDuration="3.503054948s" totalImagesPullingTime="1.721581252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:18:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:18:57.501527599 +0000 UTC m=+454.374175720" watchObservedRunningTime="2026-06-12 17:18:57.503054948 +0000 UTC m=+454.375703056" Jun 12 17:18:57.730084 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.730022 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:57.885601 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.885571 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24qmj\" (UniqueName: \"kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj\") pod \"94355930-b4ed-49b1-b4d3-5485efde8d36\" (UID: \"94355930-b4ed-49b1-b4d3-5485efde8d36\") " Jun 12 17:18:57.887968 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.887936 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj" (OuterVolumeSpecName: "kube-api-access-24qmj") pod "94355930-b4ed-49b1-b4d3-5485efde8d36" (UID: "94355930-b4ed-49b1-b4d3-5485efde8d36"). InnerVolumeSpecName "kube-api-access-24qmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:18:57.986525 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:57.986433 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24qmj\" (UniqueName: \"kubernetes.io/projected/94355930-b4ed-49b1-b4d3-5485efde8d36-kube-api-access-24qmj\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:18:58.481210 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.481179 2544 generic.go:356] "Generic (PLEG): container finished" podID="94355930-b4ed-49b1-b4d3-5485efde8d36" containerID="d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76" exitCode=0 Jun 12 17:18:58.481670 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.481226 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-55g46" Jun 12 17:18:58.481670 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.481270 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-55g46" event={"ID":"94355930-b4ed-49b1-b4d3-5485efde8d36","Type":"ContainerDied","Data":"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76"} Jun 12 17:18:58.481670 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.481332 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-55g46" event={"ID":"94355930-b4ed-49b1-b4d3-5485efde8d36","Type":"ContainerDied","Data":"6383ad81326ed95fc0f1d61a18e50cc317cac3959becf211cb7bd1a0326a7ba2"} Jun 12 17:18:58.481670 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.481354 2544 scope.go:117] "RemoveContainer" containerID="d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76" Jun 12 17:18:58.491523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.491500 2544 scope.go:117] "RemoveContainer" containerID="d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76" Jun 12 17:18:58.491830 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:18:58.491810 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76\": container with ID starting with d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76 not found: ID does not exist" containerID="d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76" Jun 12 17:18:58.491875 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.491839 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76"} err="failed to get container status \"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76\": rpc error: code = NotFound desc = could not find container \"d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76\": container with ID starting with d0da7dd46e4119a7177e8586dba7308a61042045dc8cb3875ddce847edaf8a76 not found: ID does not exist" Jun 12 17:18:58.501440 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.501407 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:58.505886 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:58.505850 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-55g46"] Jun 12 17:18:59.601833 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:18:59.601803 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94355930-b4ed-49b1-b4d3-5485efde8d36" path="/var/lib/kubelet/pods/94355930-b4ed-49b1-b4d3-5485efde8d36/volumes" Jun 12 17:19:04.588893 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:04.588853 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:19:04.589381 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:04.588944 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:19:04.611815 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:04.611784 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:19:05.533915 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:05.533888 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-82dj6" Jun 12 17:19:19.456950 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.456908 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg"] Jun 12 17:19:19.457393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.457243 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94355930-b4ed-49b1-b4d3-5485efde8d36" containerName="registry-server" Jun 12 17:19:19.457393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.457255 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="94355930-b4ed-49b1-b4d3-5485efde8d36" containerName="registry-server" Jun 12 17:19:19.457393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.457345 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="94355930-b4ed-49b1-b4d3-5485efde8d36" containerName="registry-server" Jun 12 17:19:19.461229 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.461209 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:19.463671 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.463647 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jun 12 17:19:19.463798 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.463654 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-v2f5d\"" Jun 12 17:19:19.472261 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.472236 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg"] Jun 12 17:19:19.569927 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.569892 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bl29d\" (UniqueName: \"kubernetes.io/projected/27787e06-1ed6-49b8-8376-1a1871597bc5-kube-api-access-bl29d\") pod \"dns-operator-controller-manager-864dd78c4c-cscdg\" (UID: \"27787e06-1ed6-49b8-8376-1a1871597bc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:19.671202 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.671163 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bl29d\" (UniqueName: \"kubernetes.io/projected/27787e06-1ed6-49b8-8376-1a1871597bc5-kube-api-access-bl29d\") pod \"dns-operator-controller-manager-864dd78c4c-cscdg\" (UID: \"27787e06-1ed6-49b8-8376-1a1871597bc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:19.680812 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.680781 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bl29d\" (UniqueName: \"kubernetes.io/projected/27787e06-1ed6-49b8-8376-1a1871597bc5-kube-api-access-bl29d\") pod \"dns-operator-controller-manager-864dd78c4c-cscdg\" (UID: \"27787e06-1ed6-49b8-8376-1a1871597bc5\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:19.771988 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.771887 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:19.906524 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:19.906495 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg"] Jun 12 17:19:19.909171 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:19:19.909143 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27787e06_1ed6_49b8_8376_1a1871597bc5.slice/crio-660ed5bab8a2e2333a0412abd8f38bb762dbea0562190d35c0952462da8846f2 WatchSource:0}: Error finding container 660ed5bab8a2e2333a0412abd8f38bb762dbea0562190d35c0952462da8846f2: Status 404 returned error can't find the container with id 660ed5bab8a2e2333a0412abd8f38bb762dbea0562190d35c0952462da8846f2 Jun 12 17:19:20.579192 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:20.579156 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" event={"ID":"27787e06-1ed6-49b8-8376-1a1871597bc5","Type":"ContainerStarted","Data":"660ed5bab8a2e2333a0412abd8f38bb762dbea0562190d35c0952462da8846f2"} Jun 12 17:19:22.589360 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:22.589242 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" event={"ID":"27787e06-1ed6-49b8-8376-1a1871597bc5","Type":"ContainerStarted","Data":"63e93cb2a6b5901cc36a22340a6a6e7ded3279469e9eb5de4f38bdec5f0efbe0"} Jun 12 17:19:22.589360 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:22.589345 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:22.609935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:22.609881 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" podStartSLOduration=1.187597777 podStartE2EDuration="3.609865694s" totalImagesPullingTime="2.422267917s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:19:22.608117799 +0000 UTC m=+479.480765908" watchObservedRunningTime="2026-06-12 17:19:22.609865694 +0000 UTC m=+479.482513838" Jun 12 17:19:24.855207 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.855172 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42"] Jun 12 17:19:24.858234 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.858214 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:24.860650 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.860626 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-kjkjf\"" Jun 12 17:19:24.877984 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.877951 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42"] Jun 12 17:19:24.919578 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.919533 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-777lh\" (UniqueName: \"kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:24.919770 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:24.919613 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.020195 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.020146 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-777lh\" (UniqueName: \"kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.020397 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.020235 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.020615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.020599 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.030732 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.030704 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-777lh\" (UniqueName: \"kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.168719 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.168622 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:25.319662 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.319627 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42"] Jun 12 17:19:25.324520 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:19:25.324489 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49739789_e79b_4cef_b6f9_23b635870cbc.slice/crio-68df3a5d2a87e2d450ab5920eabb5112da7dac7f6607925a555c956b47122bdf WatchSource:0}: Error finding container 68df3a5d2a87e2d450ab5920eabb5112da7dac7f6607925a555c956b47122bdf: Status 404 returned error can't find the container with id 68df3a5d2a87e2d450ab5920eabb5112da7dac7f6607925a555c956b47122bdf Jun 12 17:19:25.603466 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:25.603433 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" event={"ID":"49739789-e79b-4cef-b6f9-23b635870cbc","Type":"ContainerStarted","Data":"68df3a5d2a87e2d450ab5920eabb5112da7dac7f6607925a555c956b47122bdf"} Jun 12 17:19:32.635141 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:32.635077 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" event={"ID":"49739789-e79b-4cef-b6f9-23b635870cbc","Type":"ContainerStarted","Data":"d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12"} Jun 12 17:19:32.635591 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:32.635277 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:32.654558 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:32.654450 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" podStartSLOduration=1.577264655 podStartE2EDuration="8.65443428s" totalImagesPullingTime="7.077169625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:19:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:19:32.653644219 +0000 UTC m=+489.526292349" watchObservedRunningTime="2026-06-12 17:19:32.65443428 +0000 UTC m=+489.527082388" Jun 12 17:19:33.602847 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:33.602819 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-cscdg" Jun 12 17:19:43.642135 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:43.642100 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.376285 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.376248 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42"] Jun 12 17:19:45.376724 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.376479 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" containerName="manager" containerID="cri-o://d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12" gracePeriod=2 Jun 12 17:19:45.384199 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.384173 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42"] Jun 12 17:19:45.388134 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:19:45.388090 2544 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"49739789-e79b-4cef-b6f9-23b635870cbc\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-m4m42\": pods \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.401653 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.401623 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:19:45.402111 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.402088 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" containerName="manager" Jun 12 17:19:45.402111 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.402108 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" containerName="manager" Jun 12 17:19:45.402248 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.402187 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" containerName="manager" Jun 12 17:19:45.405021 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.405003 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.406792 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.406770 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.406890 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.406842 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tktbz\" (UniqueName: \"kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.407388 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:19:45.407353 2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" is forbidden: User \"system:node:ip-10-0-128-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-121.ec2.internal' and this object" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.424025 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.423985 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:19:45.432913 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.432875 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v"] Jun 12 17:19:45.436650 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.436629 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:45.439278 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.439260 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-4mphj\"" Jun 12 17:19:45.451267 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.451194 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v"] Jun 12 17:19:45.455577 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:19:45.455539 2544 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-m4m42\" is forbidden: User \"system:node:ip-10-0-128-121.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-121.ec2.internal' and this object" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.507637 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.507607 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.507794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.507690 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tktbz\" (UniqueName: \"kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.507794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.507721 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xsh2m\" (UniqueName: \"kubernetes.io/projected/7b80fc63-1eae-4910-a932-ce76ea86d36b-kube-api-access-xsh2m\") pod \"limitador-operator-controller-manager-859d7b78bd-jq55v\" (UID: \"7b80fc63-1eae-4910-a932-ce76ea86d36b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:45.508095 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.508071 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.519356 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.519264 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tktbz\" (UniqueName: \"kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz\") pod \"kuadrant-operator-controller-manager-8684c9ff49-m46tt\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.608265 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.608243 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xsh2m\" (UniqueName: \"kubernetes.io/projected/7b80fc63-1eae-4910-a932-ce76ea86d36b-kube-api-access-xsh2m\") pod \"limitador-operator-controller-manager-859d7b78bd-jq55v\" (UID: \"7b80fc63-1eae-4910-a932-ce76ea86d36b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:45.621920 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.621884 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xsh2m\" (UniqueName: \"kubernetes.io/projected/7b80fc63-1eae-4910-a932-ce76ea86d36b-kube-api-access-xsh2m\") pod \"limitador-operator-controller-manager-859d7b78bd-jq55v\" (UID: \"7b80fc63-1eae-4910-a932-ce76ea86d36b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:45.635289 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.635236 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.691478 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.691437 2544 generic.go:356] "Generic (PLEG): container finished" podID="49739789-e79b-4cef-b6f9-23b635870cbc" containerID="d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12" exitCode=0 Jun 12 17:19:45.691478 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.691485 2544 scope.go:117] "RemoveContainer" containerID="d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12" Jun 12 17:19:45.692033 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.691488 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m4m42" Jun 12 17:19:45.699807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.699786 2544 scope.go:117] "RemoveContainer" containerID="d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12" Jun 12 17:19:45.700055 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:19:45.700037 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12\": container with ID starting with d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12 not found: ID does not exist" containerID="d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12" Jun 12 17:19:45.700101 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.700063 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12"} err="failed to get container status \"d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12\": rpc error: code = NotFound desc = could not find container \"d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12\": container with ID starting with d56cf7ecc9468d79befadfee463ab3413f3b83ecd1430994abf5bf426e31ca12 not found: ID does not exist" Jun 12 17:19:45.709433 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.709411 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-777lh\" (UniqueName: \"kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh\") pod \"49739789-e79b-4cef-b6f9-23b635870cbc\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " Jun 12 17:19:45.709549 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.709444 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume\") pod \"49739789-e79b-4cef-b6f9-23b635870cbc\" (UID: \"49739789-e79b-4cef-b6f9-23b635870cbc\") " Jun 12 17:19:45.709930 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.709906 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "49739789-e79b-4cef-b6f9-23b635870cbc" (UID: "49739789-e79b-4cef-b6f9-23b635870cbc"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:19:45.711660 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.711635 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh" (OuterVolumeSpecName: "kube-api-access-777lh") pod "49739789-e79b-4cef-b6f9-23b635870cbc" (UID: "49739789-e79b-4cef-b6f9-23b635870cbc"). InnerVolumeSpecName "kube-api-access-777lh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:19:45.754435 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.754401 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:45.761664 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.761637 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:45.810532 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.810497 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-777lh\" (UniqueName: \"kubernetes.io/projected/49739789-e79b-4cef-b6f9-23b635870cbc-kube-api-access-777lh\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:19:45.810661 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.810537 2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/49739789-e79b-4cef-b6f9-23b635870cbc-extensions-socket-volume\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:19:45.914637 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.914604 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:19:45.916443 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:19:45.916417 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb5d02b6_5096_438e_83c4_6f2cda165d3a.slice/crio-d85bd0c4589d6a76fd8955bc599a0caff2a1bd13ff8ab3d14e081524e79a5b23 WatchSource:0}: Error finding container d85bd0c4589d6a76fd8955bc599a0caff2a1bd13ff8ab3d14e081524e79a5b23: Status 404 returned error can't find the container with id d85bd0c4589d6a76fd8955bc599a0caff2a1bd13ff8ab3d14e081524e79a5b23 Jun 12 17:19:45.940057 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:45.940031 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v"] Jun 12 17:19:45.952582 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:19:45.952554 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b80fc63_1eae_4910_a932_ce76ea86d36b.slice/crio-4d93b88b125660ffd39d99341f32072010725db02868e042be2f1aa23a8f032e WatchSource:0}: Error finding container 4d93b88b125660ffd39d99341f32072010725db02868e042be2f1aa23a8f032e: Status 404 returned error can't find the container with id 4d93b88b125660ffd39d99341f32072010725db02868e042be2f1aa23a8f032e Jun 12 17:19:46.699065 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:46.698956 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" event={"ID":"fb5d02b6-5096-438e-83c4-6f2cda165d3a","Type":"ContainerStarted","Data":"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd"} Jun 12 17:19:46.699065 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:46.699011 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:46.699065 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:46.699037 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" event={"ID":"fb5d02b6-5096-438e-83c4-6f2cda165d3a","Type":"ContainerStarted","Data":"d85bd0c4589d6a76fd8955bc599a0caff2a1bd13ff8ab3d14e081524e79a5b23"} Jun 12 17:19:46.700255 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:46.700226 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" event={"ID":"7b80fc63-1eae-4910-a932-ce76ea86d36b","Type":"ContainerStarted","Data":"4d93b88b125660ffd39d99341f32072010725db02868e042be2f1aa23a8f032e"} Jun 12 17:19:46.736360 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:46.736281 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" podStartSLOduration=1.736258998 podStartE2EDuration="1.736258998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:19:46.734965524 +0000 UTC m=+503.607613660" watchObservedRunningTime="2026-06-12 17:19:46.736258998 +0000 UTC m=+503.608907109" Jun 12 17:19:47.604487 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:47.604433 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49739789-e79b-4cef-b6f9-23b635870cbc" path="/var/lib/kubelet/pods/49739789-e79b-4cef-b6f9-23b635870cbc/volumes" Jun 12 17:19:48.715440 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:48.715407 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" event={"ID":"7b80fc63-1eae-4910-a932-ce76ea86d36b","Type":"ContainerStarted","Data":"d88599264dbb0dc0288489028ff7d4baa290b28aa925d1a0f4f84aaa1f6d1e91"} Jun 12 17:19:48.715839 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:48.715516 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:19:48.737122 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:48.737071 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" podStartSLOduration=1.998993927 podStartE2EDuration="3.737056424s" totalImagesPullingTime="1.738062497s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:19:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:19:48.73593693 +0000 UTC m=+505.608585070" watchObservedRunningTime="2026-06-12 17:19:48.737056424 +0000 UTC m=+505.609704533" Jun 12 17:19:57.707745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:57.707712 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:19:59.721661 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:19:59.721631 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-jq55v" Jun 12 17:20:01.802558 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:01.802524 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:20:01.802924 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:01.802722 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" podUID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" containerName="manager" containerID="cri-o://17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd" gracePeriod=10 Jun 12 17:20:02.061070 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.061044 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:20:02.154254 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.154219 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume\") pod \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " Jun 12 17:20:02.154452 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.154263 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tktbz\" (UniqueName: \"kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz\") pod \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\" (UID: \"fb5d02b6-5096-438e-83c4-6f2cda165d3a\") " Jun 12 17:20:02.154630 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.154605 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fb5d02b6-5096-438e-83c4-6f2cda165d3a" (UID: "fb5d02b6-5096-438e-83c4-6f2cda165d3a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:20:02.156607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.156583 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz" (OuterVolumeSpecName: "kube-api-access-tktbz") pod "fb5d02b6-5096-438e-83c4-6f2cda165d3a" (UID: "fb5d02b6-5096-438e-83c4-6f2cda165d3a"). InnerVolumeSpecName "kube-api-access-tktbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:20:02.255933 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.255838 2544 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fb5d02b6-5096-438e-83c4-6f2cda165d3a-extensions-socket-volume\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:20:02.255933 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.255869 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tktbz\" (UniqueName: \"kubernetes.io/projected/fb5d02b6-5096-438e-83c4-6f2cda165d3a-kube-api-access-tktbz\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:20:02.780157 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.780119 2544 generic.go:356] "Generic (PLEG): container finished" podID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" containerID="17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd" exitCode=0 Jun 12 17:20:02.780403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.780164 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" Jun 12 17:20:02.780403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.780209 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" event={"ID":"fb5d02b6-5096-438e-83c4-6f2cda165d3a","Type":"ContainerDied","Data":"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd"} Jun 12 17:20:02.780403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.780264 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt" event={"ID":"fb5d02b6-5096-438e-83c4-6f2cda165d3a","Type":"ContainerDied","Data":"d85bd0c4589d6a76fd8955bc599a0caff2a1bd13ff8ab3d14e081524e79a5b23"} Jun 12 17:20:02.780403 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.780317 2544 scope.go:117] "RemoveContainer" containerID="17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd" Jun 12 17:20:02.789721 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.789704 2544 scope.go:117] "RemoveContainer" containerID="17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd" Jun 12 17:20:02.789988 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:20:02.789969 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd\": container with ID starting with 17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd not found: ID does not exist" containerID="17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd" Jun 12 17:20:02.790038 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.789997 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd"} err="failed to get container status \"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd\": rpc error: code = NotFound desc = could not find container \"17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd\": container with ID starting with 17baec77ce1388d176552aba1c8694c88ee1d269f328869a858ad0c11a563cbd not found: ID does not exist" Jun 12 17:20:02.808018 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.807972 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:20:02.815101 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:02.815073 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-m46tt"] Jun 12 17:20:03.602413 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:03.602376 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" path="/var/lib/kubelet/pods/fb5d02b6-5096-438e-83c4-6f2cda165d3a/volumes" Jun 12 17:20:18.201171 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.201081 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv"] Jun 12 17:20:18.201611 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.201445 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" containerName="manager" Jun 12 17:20:18.201611 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.201457 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" containerName="manager" Jun 12 17:20:18.201611 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.201514 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb5d02b6-5096-438e-83c4-6f2cda165d3a" containerName="manager" Jun 12 17:20:18.204495 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.204478 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.207404 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.207381 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-f775d\"" Jun 12 17:20:18.218640 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.218610 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv"] Jun 12 17:20:18.289787 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289741 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.289787 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289784 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290054 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289804 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290054 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289872 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxf2s\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-kube-api-access-cxf2s\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290054 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289930 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290054 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.289985 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290054 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.290004 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290337 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.290077 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e658c-3eba-42e0-8aad-e3f82505bab6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.290337 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.290146 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.390760 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.390726 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.390944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.390779 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.390944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.390895 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.390946 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e658c-3eba-42e0-8aad-e3f82505bab6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.390985 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391031 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391072 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391242 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391216 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391326 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391236 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-data\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391387 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391349 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391387 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391382 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxf2s\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-kube-api-access-cxf2s\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391494 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391452 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-credential-socket\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391684 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391659 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d11e658c-3eba-42e0-8aad-e3f82505bab6-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.391769 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.391683 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-workload-certs\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.393384 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.393361 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-envoy\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.393850 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.393827 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-podinfo\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.403882 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.403856 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-istio-token\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.404514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.404492 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxf2s\" (UniqueName: \"kubernetes.io/projected/d11e658c-3eba-42e0-8aad-e3f82505bab6-kube-api-access-cxf2s\") pod \"maas-default-gateway-openshift-default-8559cd5744-6vdsv\" (UID: \"d11e658c-3eba-42e0-8aad-e3f82505bab6\") " pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.514785 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.514684 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:18.659854 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.659823 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv"] Jun 12 17:20:18.661871 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:20:18.661845 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd11e658c_3eba_42e0_8aad_e3f82505bab6.slice/crio-a2f108a3d534a217eee9b7074252fc20a75eef62e51c54c1e0f546f76a737378 WatchSource:0}: Error finding container a2f108a3d534a217eee9b7074252fc20a75eef62e51c54c1e0f546f76a737378: Status 404 returned error can't find the container with id a2f108a3d534a217eee9b7074252fc20a75eef62e51c54c1e0f546f76a737378 Jun 12 17:20:18.663819 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.663788 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:20:18.663932 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.663863 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:20:18.663932 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.663892 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:20:18.853742 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.853702 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" event={"ID":"d11e658c-3eba-42e0-8aad-e3f82505bab6","Type":"ContainerStarted","Data":"071fcc89c4668158eb560b5f568ae5371ca50ff313d4316d581538066ce69e51"} Jun 12 17:20:18.853944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.853761 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" event={"ID":"d11e658c-3eba-42e0-8aad-e3f82505bab6","Type":"ContainerStarted","Data":"a2f108a3d534a217eee9b7074252fc20a75eef62e51c54c1e0f546f76a737378"} Jun 12 17:20:18.876744 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:18.876669 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" podStartSLOduration=0.876647608 podStartE2EDuration="876.647608ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:20:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:20:18.874502296 +0000 UTC m=+535.747150426" watchObservedRunningTime="2026-06-12 17:20:18.876647608 +0000 UTC m=+535.749295719" Jun 12 17:20:19.515336 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:19.515280 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:19.520396 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:19.520373 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:19.858560 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:19.858531 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:19.859587 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:19.859548 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:20:20.055551 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.055514 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:20:20.060734 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.060709 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.063149 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.063128 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jun 12 17:20:20.063443 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.063427 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bzv5f\"" Jun 12 17:20:20.072676 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.072651 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:20:20.106863 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.106829 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:20:20.208316 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.208207 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f862d37-568a-4cd8-95d2-80bd84e23b50-config-file\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.208482 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.208366 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7znl8\" (UniqueName: \"kubernetes.io/projected/3f862d37-568a-4cd8-95d2-80bd84e23b50-kube-api-access-7znl8\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.309157 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.309115 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7znl8\" (UniqueName: \"kubernetes.io/projected/3f862d37-568a-4cd8-95d2-80bd84e23b50-kube-api-access-7znl8\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.309400 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.309179 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f862d37-568a-4cd8-95d2-80bd84e23b50-config-file\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.309794 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.309773 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3f862d37-568a-4cd8-95d2-80bd84e23b50-config-file\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.319246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.319225 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7znl8\" (UniqueName: \"kubernetes.io/projected/3f862d37-568a-4cd8-95d2-80bd84e23b50-kube-api-access-7znl8\") pod \"limitador-limitador-54d6d988d5-6qbxg\" (UID: \"3f862d37-568a-4cd8-95d2-80bd84e23b50\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.372319 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.372266 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:20.511233 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.511203 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:20:20.513753 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:20:20.513721 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f862d37_568a_4cd8_95d2_80bd84e23b50.slice/crio-68f71be14f41c2e29608217bc92b74e3c592b71190a113013c020d6d21026143 WatchSource:0}: Error finding container 68f71be14f41c2e29608217bc92b74e3c592b71190a113013c020d6d21026143: Status 404 returned error can't find the container with id 68f71be14f41c2e29608217bc92b74e3c592b71190a113013c020d6d21026143 Jun 12 17:20:20.863903 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:20.863871 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" event={"ID":"3f862d37-568a-4cd8-95d2-80bd84e23b50","Type":"ContainerStarted","Data":"68f71be14f41c2e29608217bc92b74e3c592b71190a113013c020d6d21026143"} Jun 12 17:20:23.881432 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:23.881397 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" event={"ID":"3f862d37-568a-4cd8-95d2-80bd84e23b50","Type":"ContainerStarted","Data":"963bfd4d5b24f37efd5bc7fd68e05d0118ba5507159336075039f7a898eae231"} Jun 12 17:20:23.881823 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:23.881526 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:20:23.904526 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:23.904471 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" podStartSLOduration=1.327321574 podStartE2EDuration="3.904454722s" totalImagesPullingTime="2.577133148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:20:23.90133091 +0000 UTC m=+540.773979046" watchObservedRunningTime="2026-06-12 17:20:23.904454722 +0000 UTC m=+540.777102830" Jun 12 17:20:34.887319 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:20:34.887260 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-6qbxg" Jun 12 17:21:23.572459 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:23.572431 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:21:23.573020 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:23.572545 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:21:47.524494 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.524410 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:21:47.527671 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.527654 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.529972 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.529946 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-sk45x\"" Jun 12 17:21:47.529972 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.529966 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 12 17:21:47.530147 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.529947 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jun 12 17:21:47.536783 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.536704 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:21:47.589815 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.589781 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpbwb\" (UniqueName: \"kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.590018 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.589844 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.691395 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.691358 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpbwb\" (UniqueName: \"kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.691585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.691419 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.694082 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.694056 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.701326 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.701284 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpbwb\" (UniqueName: \"kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb\") pod \"maas-controller-765bfcd6bd-vrvhh\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.840224 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.840136 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:47.978085 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:47.978048 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:21:47.982091 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:21:47.982063 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a26442e_e196_4aa1_bcac_6744914f7c13.slice/crio-a700bdebde05257306cc1db287a104456f83c83aee8356f0a224295875560381 WatchSource:0}: Error finding container a700bdebde05257306cc1db287a104456f83c83aee8356f0a224295875560381: Status 404 returned error can't find the container with id a700bdebde05257306cc1db287a104456f83c83aee8356f0a224295875560381 Jun 12 17:21:48.261141 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:48.261106 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" event={"ID":"9a26442e-e196-4aa1-bcac-6744914f7c13","Type":"ContainerStarted","Data":"a700bdebde05257306cc1db287a104456f83c83aee8356f0a224295875560381"} Jun 12 17:21:51.279135 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:51.279094 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" event={"ID":"9a26442e-e196-4aa1-bcac-6744914f7c13","Type":"ContainerStarted","Data":"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f"} Jun 12 17:21:51.279783 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:51.279164 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:21:51.297827 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:21:51.297776 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" podStartSLOduration=1.884992568 podStartE2EDuration="4.297761251s" totalImagesPullingTime="2.412768683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:21:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:21:51.295965944 +0000 UTC m=+628.168614054" watchObservedRunningTime="2026-06-12 17:21:51.297761251 +0000 UTC m=+628.170409360" Jun 12 17:22:02.291273 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:02.291243 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:22:21.771436 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.771397 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-86b7d85b46-rgdwt"] Jun 12 17:22:21.775492 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.775473 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.777463 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.777442 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 12 17:22:21.777843 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.777829 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-wp58g\"" Jun 12 17:22:21.783079 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.782815 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-86b7d85b46-rgdwt"] Jun 12 17:22:21.891554 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.891518 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-maas-api-tls\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.891733 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.891642 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-openshift-service-ca\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.891733 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.891688 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxnvn\" (UniqueName: \"kubernetes.io/projected/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-kube-api-access-hxnvn\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.992919 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.992887 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-maas-api-tls\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.993124 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.992953 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-openshift-service-ca\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.993124 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.992981 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxnvn\" (UniqueName: \"kubernetes.io/projected/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-kube-api-access-hxnvn\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.993703 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.993676 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-openshift-service-ca\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:21.995518 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:21.995498 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-maas-api-tls\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:22.003570 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.003544 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxnvn\" (UniqueName: \"kubernetes.io/projected/d6bef020-e5d7-4821-bbf6-546fdaaa2f10-kube-api-access-hxnvn\") pod \"maas-api-86b7d85b46-rgdwt\" (UID: \"d6bef020-e5d7-4821-bbf6-546fdaaa2f10\") " pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:22.087074 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.086986 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:22.236523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.236498 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-86b7d85b46-rgdwt"] Jun 12 17:22:22.238856 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:22:22.238827 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6bef020_e5d7_4821_bbf6_546fdaaa2f10.slice/crio-5c1f9386a85f2a46eb7a8d2e8fb88408c3735204c3f457f4db8e41babac04a86 WatchSource:0}: Error finding container 5c1f9386a85f2a46eb7a8d2e8fb88408c3735204c3f457f4db8e41babac04a86: Status 404 returned error can't find the container with id 5c1f9386a85f2a46eb7a8d2e8fb88408c3735204c3f457f4db8e41babac04a86 Jun 12 17:22:22.240211 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.240195 2544 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 12 17:22:22.419745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.419706 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-86b7d85b46-rgdwt" event={"ID":"d6bef020-e5d7-4821-bbf6-546fdaaa2f10","Type":"ContainerStarted","Data":"5c1f9386a85f2a46eb7a8d2e8fb88408c3735204c3f457f4db8e41babac04a86"} Jun 12 17:22:22.423930 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:22.423900 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:22:24.453585 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:24.453545 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-86b7d85b46-rgdwt" event={"ID":"d6bef020-e5d7-4821-bbf6-546fdaaa2f10","Type":"ContainerStarted","Data":"55109d7605935016a61ebefc8d43166da950dca6d6a8e1b22dc44ff99b0636b4"} Jun 12 17:22:24.453972 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:24.453628 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:24.472467 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:24.472421 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-86b7d85b46-rgdwt" podStartSLOduration=1.739897805 podStartE2EDuration="3.472405076s" totalImagesPullingTime="1.732507271s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:22:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:22:24.470758977 +0000 UTC m=+661.343407085" watchObservedRunningTime="2026-06-12 17:22:24.472405076 +0000 UTC m=+661.345053185" Jun 12 17:22:30.463108 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.463079 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-86b7d85b46-rgdwt" Jun 12 17:22:30.482915 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.482885 2544 generic.go:356] "Generic (PLEG): container finished" podID="d11e658c-3eba-42e0-8aad-e3f82505bab6" containerID="071fcc89c4668158eb560b5f568ae5371ca50ff313d4316d581538066ce69e51" exitCode=0 Jun 12 17:22:30.483073 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.482962 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" event={"ID":"d11e658c-3eba-42e0-8aad-e3f82505bab6","Type":"ContainerDied","Data":"071fcc89c4668158eb560b5f568ae5371ca50ff313d4316d581538066ce69e51"} Jun 12 17:22:30.483376 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.483356 2544 scope.go:117] "RemoveContainer" containerID="071fcc89c4668158eb560b5f568ae5371ca50ff313d4316d581538066ce69e51" Jun 12 17:22:30.484471 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.484437 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:22:30.484539 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.484512 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:22:30.484578 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:30.484563 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:22:31.489818 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:31.489780 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" event={"ID":"d11e658c-3eba-42e0-8aad-e3f82505bab6","Type":"ContainerStarted","Data":"068e2579f13557a96aa853f775f11d5b8327910286bce9e30517d0f519d23655"} Jun 12 17:22:31.514928 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:31.514894 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:22:31.516093 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:31.516056 2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-6vdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jun 12 17:22:31.516223 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:31.516148 2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" podUID="d11e658c-3eba-42e0-8aad-e3f82505bab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jun 12 17:22:32.515650 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:32.515609 2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-6vdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jun 12 17:22:32.516169 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:32.515674 2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" podUID="d11e658c-3eba-42e0-8aad-e3f82505bab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jun 12 17:22:33.515373 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:33.515335 2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-6vdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jun 12 17:22:33.515759 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:33.515727 2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" podUID="d11e658c-3eba-42e0-8aad-e3f82505bab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jun 12 17:22:34.515612 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:34.515573 2544 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-8559cd5744-6vdsv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jun 12 17:22:34.515821 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:34.515651 2544 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" podUID="d11e658c-3eba-42e0-8aad-e3f82505bab6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jun 12 17:22:34.858850 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:34.858816 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:22:35.519459 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:35.519421 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:22:35.520567 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:22:35.520544 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-8559cd5744-6vdsv" Jun 12 17:25:40.496371 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.496337 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj"] Jun 12 17:25:40.498743 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.498723 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.501045 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.501022 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 12 17:25:40.501216 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.501022 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 12 17:25:40.501539 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.501521 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jun 12 17:25:40.501631 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.501521 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-k4wsf\"" Jun 12 17:25:40.509031 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.509003 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj"] Jun 12 17:25:40.626242 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626198 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.626242 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626243 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.626523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626267 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.626523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626373 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.626523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626435 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6c85ea-9447-40fc-9674-72a4ad1556eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.626523 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.626459 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbgr6\" (UniqueName: \"kubernetes.io/projected/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kube-api-access-mbgr6\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727069 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727033 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6c85ea-9447-40fc-9674-72a4ad1556eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727227 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727075 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbgr6\" (UniqueName: \"kubernetes.io/projected/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kube-api-access-mbgr6\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727227 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727148 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727227 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727176 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727227 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727209 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727451 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727248 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727667 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-home\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727710 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.727844 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.727739 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-model-cache\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.729645 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.729615 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4d6c85ea-9447-40fc-9674-72a4ad1556eb-dshm\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.729802 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.729785 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4d6c85ea-9447-40fc-9674-72a4ad1556eb-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.735721 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.735693 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbgr6\" (UniqueName: \"kubernetes.io/projected/4d6c85ea-9447-40fc-9674-72a4ad1556eb-kube-api-access-mbgr6\") pod \"e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj\" (UID: \"4d6c85ea-9447-40fc-9674-72a4ad1556eb\") " pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.780477 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.780372 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:25:40.810669 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.810631 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:40.952438 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:40.952402 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj"] Jun 12 17:25:40.955280 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:25:40.955254 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d6c85ea_9447_40fc_9674_72a4ad1556eb.slice/crio-244bd295123659500cb3a2aea673a695d0696633fd5e3ed0488ffdc3adc4a6b0 WatchSource:0}: Error finding container 244bd295123659500cb3a2aea673a695d0696633fd5e3ed0488ffdc3adc4a6b0: Status 404 returned error can't find the container with id 244bd295123659500cb3a2aea673a695d0696633fd5e3ed0488ffdc3adc4a6b0 Jun 12 17:25:41.381446 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:41.381407 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" event={"ID":"4d6c85ea-9447-40fc-9674-72a4ad1556eb","Type":"ContainerStarted","Data":"244bd295123659500cb3a2aea673a695d0696633fd5e3ed0488ffdc3adc4a6b0"} Jun 12 17:25:45.402284 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:45.402249 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" event={"ID":"4d6c85ea-9447-40fc-9674-72a4ad1556eb","Type":"ContainerStarted","Data":"bf576cf2ea92c4912da6a22616a7afc67a77867d11cc3a685f537932ff94c54f"} Jun 12 17:25:45.991149 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:45.991106 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm"] Jun 12 17:25:45.995017 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:45.994994 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:45.997323 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:45.997272 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 12 17:25:46.004277 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.004250 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm"] Jun 12 17:25:46.073888 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.073843 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.073888 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.073884 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.074097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.073951 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwsbt\" (UniqueName: \"kubernetes.io/projected/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kube-api-access-xwsbt\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.074097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.073989 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.074097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.074008 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.074097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.074027 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b0c97-21ce-4f48-bfca-54aa42ae72ae-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175375 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175329 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175375 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175374 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175582 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175408 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b0c97-21ce-4f48-bfca-54aa42ae72ae-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175582 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175500 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175582 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175520 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175688 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175586 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwsbt\" (UniqueName: \"kubernetes.io/projected/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kube-api-access-xwsbt\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175779 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-home\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.175996 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175889 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.176122 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.175931 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-model-cache\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.178422 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.178397 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/185b0c97-21ce-4f48-bfca-54aa42ae72ae-tls-certs\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.178536 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.178479 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/185b0c97-21ce-4f48-bfca-54aa42ae72ae-dshm\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.187806 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.187601 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwsbt\" (UniqueName: \"kubernetes.io/projected/185b0c97-21ce-4f48-bfca-54aa42ae72ae-kube-api-access-xwsbt\") pod \"e2e-distinct-simulated-kserve-5cdff5474d-b97vm\" (UID: \"185b0c97-21ce-4f48-bfca-54aa42ae72ae\") " pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.305606 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.305512 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:25:46.447099 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.447074 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm"] Jun 12 17:25:46.449619 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:25:46.449591 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185b0c97_21ce_4f48_bfca_54aa42ae72ae.slice/crio-ff0be5ef26a96ac7ed4d4df39f4933ea83d54c25eeb68f07da9e81b304858100 WatchSource:0}: Error finding container ff0be5ef26a96ac7ed4d4df39f4933ea83d54c25eeb68f07da9e81b304858100: Status 404 returned error can't find the container with id ff0be5ef26a96ac7ed4d4df39f4933ea83d54c25eeb68f07da9e81b304858100 Jun 12 17:25:46.483852 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:46.483824 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:25:47.415318 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:47.415248 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" event={"ID":"185b0c97-21ce-4f48-bfca-54aa42ae72ae","Type":"ContainerStarted","Data":"2d1095d5b53f33ed19715e342e67af6db66e1371591beeb55a8ffb6428270bc2"} Jun 12 17:25:47.415488 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:47.415345 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" event={"ID":"185b0c97-21ce-4f48-bfca-54aa42ae72ae","Type":"ContainerStarted","Data":"ff0be5ef26a96ac7ed4d4df39f4933ea83d54c25eeb68f07da9e81b304858100"} Jun 12 17:25:48.421796 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:48.421760 2544 generic.go:356] "Generic (PLEG): container finished" podID="4d6c85ea-9447-40fc-9674-72a4ad1556eb" containerID="bf576cf2ea92c4912da6a22616a7afc67a77867d11cc3a685f537932ff94c54f" exitCode=0 Jun 12 17:25:48.422170 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:48.421832 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" event={"ID":"4d6c85ea-9447-40fc-9674-72a4ad1556eb","Type":"ContainerDied","Data":"bf576cf2ea92c4912da6a22616a7afc67a77867d11cc3a685f537932ff94c54f"} Jun 12 17:25:50.433807 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:50.433772 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" event={"ID":"4d6c85ea-9447-40fc-9674-72a4ad1556eb","Type":"ContainerStarted","Data":"4f2d07010fb840c768930f956987d0a4616562a20b8f5ecc9dc8036785c504d8"} Jun 12 17:25:50.452892 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:50.452837 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" podStartSLOduration=1.809563538 podStartE2EDuration="10.452822576s" totalImagesPullingTime="5.355331148s" totalInitContainerRuntime="3.28792789s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:25:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:25:50.452740226 +0000 UTC m=+867.325388334" watchObservedRunningTime="2026-06-12 17:25:50.452822576 +0000 UTC m=+867.325470706" Jun 12 17:25:50.811825 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:50.811734 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:50.811825 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:50.811798 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:50.824573 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:50.824539 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:51.449797 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:51.449768 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj" Jun 12 17:25:53.583103 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:25:53.583061 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:26:13.901574 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:13.901500 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm"] Jun 12 17:26:13.905128 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:13.905105 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:13.907328 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:13.907288 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 12 17:26:13.918068 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:13.918045 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm"] Jun 12 17:26:14.040212 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040169 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d941e7ee-9874-48f3-b497-e21cc223eb72-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.040435 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040256 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.040435 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040397 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.040556 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040433 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9pxv\" (UniqueName: \"kubernetes.io/projected/d941e7ee-9874-48f3-b497-e21cc223eb72-kube-api-access-j9pxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.040609 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040543 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.040609 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.040602 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141762 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141724 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141771 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141805 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d941e7ee-9874-48f3-b497-e21cc223eb72-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141830 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141895 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.141935 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.141918 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9pxv\" (UniqueName: \"kubernetes.io/projected/d941e7ee-9874-48f3-b497-e21cc223eb72-kube-api-access-j9pxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.142258 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.142230 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.142258 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.142247 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.142374 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.142274 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.144164 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.144139 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d941e7ee-9874-48f3-b497-e21cc223eb72-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.144488 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.144466 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d941e7ee-9874-48f3-b497-e21cc223eb72-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.151118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.151097 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9pxv\" (UniqueName: \"kubernetes.io/projected/d941e7ee-9874-48f3-b497-e21cc223eb72-kube-api-access-j9pxv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm\" (UID: \"d941e7ee-9874-48f3-b497-e21cc223eb72\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.240214 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.240125 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:26:14.380124 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:26:14.380097 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd941e7ee_9874_48f3_b497_e21cc223eb72.slice/crio-6dbfc7622f013f2a98a43f4f124b3d2f9642d0276a44ed997a28582802e170bd WatchSource:0}: Error finding container 6dbfc7622f013f2a98a43f4f124b3d2f9642d0276a44ed997a28582802e170bd: Status 404 returned error can't find the container with id 6dbfc7622f013f2a98a43f4f124b3d2f9642d0276a44ed997a28582802e170bd Jun 12 17:26:14.392945 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.392923 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm"] Jun 12 17:26:14.547091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.546998 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" event={"ID":"d941e7ee-9874-48f3-b497-e21cc223eb72","Type":"ContainerStarted","Data":"e3de508ae30cbdab9960ca07ec12df18deb24488af8bc73fb6460d23c39bda09"} Jun 12 17:26:14.547091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.547066 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" event={"ID":"d941e7ee-9874-48f3-b497-e21cc223eb72","Type":"ContainerStarted","Data":"6dbfc7622f013f2a98a43f4f124b3d2f9642d0276a44ed997a28582802e170bd"} Jun 12 17:26:14.687707 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:14.687672 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:26:20.790460 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.789583 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k"] Jun 12 17:26:20.795958 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.795933 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.799119 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.798413 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jun 12 17:26:20.810225 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.810196 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k"] Jun 12 17:26:20.908369 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908280 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djlr6\" (UniqueName: \"kubernetes.io/projected/814c4d44-5f5a-446f-b9f6-81eb8db54275-kube-api-access-djlr6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.908564 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908410 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.908564 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908431 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.908564 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908455 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814c4d44-5f5a-446f-b9f6-81eb8db54275-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.908735 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908566 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:20.908735 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:20.908670 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010164 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010071 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djlr6\" (UniqueName: \"kubernetes.io/projected/814c4d44-5f5a-446f-b9f6-81eb8db54275-kube-api-access-djlr6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010236 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010266 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010316 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814c4d44-5f5a-446f-b9f6-81eb8db54275-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010406 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010386 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010625 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010487 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010712 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010685 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-model-cache\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010769 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010730 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-kserve-provision-location\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.010840 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.010817 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-home\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.012737 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.012715 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/814c4d44-5f5a-446f-b9f6-81eb8db54275-dshm\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.013103 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.013081 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/814c4d44-5f5a-446f-b9f6-81eb8db54275-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.020468 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.020445 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djlr6\" (UniqueName: \"kubernetes.io/projected/814c4d44-5f5a-446f-b9f6-81eb8db54275-kube-api-access-djlr6\") pod \"facebook-opt-125m-simulated-kserve-6f969c879b-6297k\" (UID: \"814c4d44-5f5a-446f-b9f6-81eb8db54275\") " pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.117615 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.117567 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:21.260142 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.260115 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k"] Jun 12 17:26:21.262210 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:26:21.262170 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod814c4d44_5f5a_446f_b9f6_81eb8db54275.slice/crio-a77e807490366494f3fbac7a01ccd3d7275efbf82dd4a13be8934a499259b8a6 WatchSource:0}: Error finding container a77e807490366494f3fbac7a01ccd3d7275efbf82dd4a13be8934a499259b8a6: Status 404 returned error can't find the container with id a77e807490366494f3fbac7a01ccd3d7275efbf82dd4a13be8934a499259b8a6 Jun 12 17:26:21.581224 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.581121 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" event={"ID":"814c4d44-5f5a-446f-b9f6-81eb8db54275","Type":"ContainerStarted","Data":"8f5908374844fb94c44853f1d511414140af0645845e82a8718ec43772429c10"} Jun 12 17:26:21.581224 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.581193 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" event={"ID":"814c4d44-5f5a-446f-b9f6-81eb8db54275","Type":"ContainerStarted","Data":"a77e807490366494f3fbac7a01ccd3d7275efbf82dd4a13be8934a499259b8a6"} Jun 12 17:26:21.886269 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:21.886234 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:26:23.592960 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:23.592925 2544 generic.go:356] "Generic (PLEG): container finished" podID="814c4d44-5f5a-446f-b9f6-81eb8db54275" containerID="8f5908374844fb94c44853f1d511414140af0645845e82a8718ec43772429c10" exitCode=0 Jun 12 17:26:23.593469 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:23.593010 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" event={"ID":"814c4d44-5f5a-446f-b9f6-81eb8db54275","Type":"ContainerDied","Data":"8f5908374844fb94c44853f1d511414140af0645845e82a8718ec43772429c10"} Jun 12 17:26:23.604246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:23.604224 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:26:23.605368 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:23.605346 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:26:24.599661 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:24.599623 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" event={"ID":"814c4d44-5f5a-446f-b9f6-81eb8db54275","Type":"ContainerStarted","Data":"a1ad3e766c160a25e4d94d1f710d2227fe39bf838d84827769944295c53e51bd"} Jun 12 17:26:24.620916 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:24.620864 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" podStartSLOduration=3.149085571 podStartE2EDuration="4.62084833s" totalImagesPullingTime="217.982755ms" totalInitContainerRuntime="1.253780004s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:26:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:26:24.618782871 +0000 UTC m=+901.491430980" watchObservedRunningTime="2026-06-12 17:26:24.62084833 +0000 UTC m=+901.493496438" Jun 12 17:26:26.785168 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:26.785132 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:26:31.117715 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:31.117678 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:31.118136 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:31.117759 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:31.130249 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:31.130223 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:31.643737 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:31.643699 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-6f969c879b-6297k" Jun 12 17:26:42.684184 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:42.684154 2544 generic.go:356] "Generic (PLEG): container finished" podID="185b0c97-21ce-4f48-bfca-54aa42ae72ae" containerID="2d1095d5b53f33ed19715e342e67af6db66e1371591beeb55a8ffb6428270bc2" exitCode=0 Jun 12 17:26:42.684565 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:42.684209 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" event={"ID":"185b0c97-21ce-4f48-bfca-54aa42ae72ae","Type":"ContainerDied","Data":"2d1095d5b53f33ed19715e342e67af6db66e1371591beeb55a8ffb6428270bc2"} Jun 12 17:26:43.691457 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:43.691426 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" event={"ID":"185b0c97-21ce-4f48-bfca-54aa42ae72ae","Type":"ContainerStarted","Data":"ddd68a3fee05e2fa6354a2831430a2f89a2358c020f111baea13d8b83630e8d8"} Jun 12 17:26:43.711559 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:43.711506 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" podStartSLOduration=2.264493004 podStartE2EDuration="58.711490702s" totalImagesPullingTime="288.311927ms" totalInitContainerRuntime="56.158685771s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:25:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:26:43.710037065 +0000 UTC m=+920.582685192" watchObservedRunningTime="2026-06-12 17:26:43.711490702 +0000 UTC m=+920.584138801" Jun 12 17:26:46.306157 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:46.306123 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:26:46.306664 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:46.306176 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:26:46.318641 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:46.318614 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:26:46.717284 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:26:46.717254 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5cdff5474d-b97vm" Jun 12 17:27:06.806968 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:06.806934 2544 generic.go:356] "Generic (PLEG): container finished" podID="d941e7ee-9874-48f3-b497-e21cc223eb72" containerID="e3de508ae30cbdab9960ca07ec12df18deb24488af8bc73fb6460d23c39bda09" exitCode=0 Jun 12 17:27:06.807396 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:06.806973 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" event={"ID":"d941e7ee-9874-48f3-b497-e21cc223eb72","Type":"ContainerDied","Data":"e3de508ae30cbdab9960ca07ec12df18deb24488af8bc73fb6460d23c39bda09"} Jun 12 17:27:07.819775 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:07.819728 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" event={"ID":"d941e7ee-9874-48f3-b497-e21cc223eb72","Type":"ContainerStarted","Data":"5cba560d7da62aa3bf203dea0d272ddb1d5ffe6f4266f44c8628fa2d5c0b3cd8"} Jun 12 17:27:07.840367 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:07.840317 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" podStartSLOduration=2.932124246 podStartE2EDuration="54.840282793s" totalImagesPullingTime="231.889398ms" totalInitContainerRuntime="51.676269149s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:27:07.839973345 +0000 UTC m=+944.712621482" watchObservedRunningTime="2026-06-12 17:27:07.840282793 +0000 UTC m=+944.712930901" Jun 12 17:27:14.240345 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:14.240287 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:27:14.240745 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:14.240408 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:27:14.253464 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:14.253440 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:27:14.864735 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:27:14.864704 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm" Jun 12 17:28:10.288310 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:10.288255 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:28:20.588900 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:20.588862 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:28:29.391821 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:29.391779 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:28:39.584853 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:39.584817 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:28:48.990579 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:48.990536 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:28:58.774242 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:28:58.774210 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:29:07.029974 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.029938 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:29:07.030479 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.030151 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" podUID="9a26442e-e196-4aa1-bcac-6744914f7c13" containerName="manager" containerID="cri-o://823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f" gracePeriod=30 Jun 12 17:29:07.283902 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.283831 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:29:07.411939 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.411912 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert\") pod \"9a26442e-e196-4aa1-bcac-6744914f7c13\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " Jun 12 17:29:07.412278 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.412255 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpbwb\" (UniqueName: \"kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb\") pod \"9a26442e-e196-4aa1-bcac-6744914f7c13\" (UID: \"9a26442e-e196-4aa1-bcac-6744914f7c13\") " Jun 12 17:29:07.413545 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.413511 2544 generic.go:356] "Generic (PLEG): container finished" podID="9a26442e-e196-4aa1-bcac-6744914f7c13" containerID="823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f" exitCode=0 Jun 12 17:29:07.413681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.413573 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" Jun 12 17:29:07.413681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.413608 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" event={"ID":"9a26442e-e196-4aa1-bcac-6744914f7c13","Type":"ContainerDied","Data":"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f"} Jun 12 17:29:07.413681 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.413671 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-765bfcd6bd-vrvhh" event={"ID":"9a26442e-e196-4aa1-bcac-6744914f7c13","Type":"ContainerDied","Data":"a700bdebde05257306cc1db287a104456f83c83aee8356f0a224295875560381"} Jun 12 17:29:07.413832 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.413693 2544 scope.go:117] "RemoveContainer" containerID="823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f" Jun 12 17:29:07.415204 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.415176 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb" (OuterVolumeSpecName: "kube-api-access-qpbwb") pod "9a26442e-e196-4aa1-bcac-6744914f7c13" (UID: "9a26442e-e196-4aa1-bcac-6744914f7c13"). InnerVolumeSpecName "kube-api-access-qpbwb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:29:07.415353 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.415217 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9a26442e-e196-4aa1-bcac-6744914f7c13" (UID: "9a26442e-e196-4aa1-bcac-6744914f7c13"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 12 17:29:07.432837 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.432818 2544 scope.go:117] "RemoveContainer" containerID="823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f" Jun 12 17:29:07.433184 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:29:07.433161 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f\": container with ID starting with 823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f not found: ID does not exist" containerID="823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f" Jun 12 17:29:07.433236 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.433194 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f"} err="failed to get container status \"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f\": rpc error: code = NotFound desc = could not find container \"823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f\": container with ID starting with 823aad9c2b4d5cc46ff3a606ce640cde791bfd95ba255c307f1616dcf6f58a5f not found: ID does not exist" Jun 12 17:29:07.514193 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.514151 2544 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9a26442e-e196-4aa1-bcac-6744914f7c13-webhook-cert\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:29:07.514193 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.514186 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpbwb\" (UniqueName: \"kubernetes.io/projected/9a26442e-e196-4aa1-bcac-6744914f7c13-kube-api-access-qpbwb\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:29:07.730473 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.730442 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:29:07.734036 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:07.734012 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-765bfcd6bd-vrvhh"] Jun 12 17:29:09.603240 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:29:09.603209 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a26442e-e196-4aa1-bcac-6744914f7c13" path="/var/lib/kubelet/pods/9a26442e-e196-4aa1-bcac-6744914f7c13/volumes" Jun 12 17:30:13.894735 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:30:13.894697 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:30:29.095085 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:30:29.095041 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:31:08.685393 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:08.685311 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:31:23.633664 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:23.633635 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:31:23.635721 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:23.635697 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:31:25.390190 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:25.390158 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:31:39.879581 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:39.879539 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:31:56.582493 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:31:56.582455 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:32:56.480086 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:32:56.480050 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:33:05.687759 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:33:05.687720 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:33:22.189586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:33:22.189548 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:33:30.778810 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:33:30.778773 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:33:48.381683 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:33:48.381600 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:33:56.493531 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:33:56.493489 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:34:29.292260 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:34:29.292222 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:34:37.884127 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:34:37.884079 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:34:45.876484 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:34:45.876444 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:34:54.483665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:34:54.483630 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:35:02.878009 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:35:02.877968 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:35:28.486883 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:35:28.486845 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:35:40.008878 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:35:40.008839 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:36:23.663925 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:36:23.663899 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:36:23.665281 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:36:23.665263 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:36:27.580325 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:36:27.580271 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:36:35.480051 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:36:35.480016 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:36:54.603531 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:36:54.603445 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:03.080083 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:03.080039 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:12.179732 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:12.179688 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:20.776763 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:20.776718 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:29.787660 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:29.787621 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:38.369988 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:38.369945 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:47.383869 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:47.383829 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:37:55.275952 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:37:55.275910 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:38:04.080231 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:38:04.080192 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:38:12.378187 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:38:12.378109 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:40:35.088036 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:40:35.087988 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:40:40.181248 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:40:40.181204 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:04.084312 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:04.084267 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:08.373636 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:08.373597 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:16.007642 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:16.007554 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:23.693451 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:23.693422 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:41:23.695801 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:23.695781 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:41:26.280600 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:26.280563 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:35.279138 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:35.279096 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:45.368468 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:45.368431 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:41:54.678512 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:41:54.678479 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:42:04.996219 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:42:04.996177 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:42:13.683944 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:42:13.683900 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:42:24.381028 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:42:24.380992 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:42:33.596991 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:42:33.596948 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:42:38.474801 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:42:38.474758 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:43:06.480273 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:43:06.480235 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:43:49.681965 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:43:49.681924 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:43:57.481262 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:43:57.481223 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:06.680719 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:06.680676 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:14.874764 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:14.874678 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:24.080533 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:24.080494 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:36.582674 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:36.582633 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:45.672206 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:45.672166 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:50.979464 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:50.979429 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:44:57.976669 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:44:57.976627 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:45:06.079559 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:45:06.079520 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:45:14.573483 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:45:14.573438 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:45:24.885140 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:45:24.885103 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:45:43.073224 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:45:43.073138 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:45:50.773312 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:45:50.773244 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:00.179989 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:00.179948 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:08.373420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:08.373379 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:23.725286 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:23.725257 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:46:23.728288 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:23.728267 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-69swk_1ea666b9-869f-479d-a9de-d27c52931609/ovn-acl-logging/0.log" Jun 12 17:46:25.682013 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:25.681971 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:34.376163 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:34.376122 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:42.878903 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:42.878864 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:46:50.475345 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:46:50.475287 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:00.174450 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:00.174405 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:08.574954 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:08.574917 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:17.673203 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:17.673111 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:28.385343 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:28.385290 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:37.873459 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:37.873415 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:48.778631 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:48.778600 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:47:57.984864 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:47:57.984824 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:03.078039 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:03.077991 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:10.179952 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:10.179914 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:15.181528 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:15.181484 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:30.378026 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:30.377990 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:38.984555 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:38.984515 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:48.576361 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:48.576248 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:48:54.372633 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:48:54.372590 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:49:16.275928 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:16.275890 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:49:28.775087 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:28.775049 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-6qbxg"] Jun 12 17:49:42.113471 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.113434 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:42.114067 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.114043 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a26442e-e196-4aa1-bcac-6744914f7c13" containerName="manager" Jun 12 17:49:42.114067 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.114071 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a26442e-e196-4aa1-bcac-6744914f7c13" containerName="manager" Jun 12 17:49:42.114263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.114193 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a26442e-e196-4aa1-bcac-6744914f7c13" containerName="manager" Jun 12 17:49:42.117699 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.117681 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.119869 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.119844 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-622ae149-openshift-default-dockercfg-xhlhn\"" Jun 12 17:49:42.129518 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.129490 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:42.234725 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234690 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.234725 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234727 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.234942 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234817 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.234942 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234874 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9gwq\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.234942 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234906 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.234942 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234941 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.235346 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.234971 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.235346 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.235017 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.235346 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.235046 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336074 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336029 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336090 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336120 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336148 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336188 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336279 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336216 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336360 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336395 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9gwq\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336426 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336816 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336620 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336903 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336872 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.336999 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.336969 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.337132 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.337031 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.337132 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.337049 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.339169 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.339149 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.339272 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.339148 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.345531 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.345498 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.345648 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.345600 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9gwq\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq\") pod \"e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.431167 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.431069 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:42.580634 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.580601 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:42.583439 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:49:42.583403 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cc352ef_2b02_4819_a396_114c4245490d.slice/crio-9345f01976fe1b4bda4fa47dc72c6f50a6dc7608b4213f5037f673f58499aaaa WatchSource:0}: Error finding container 9345f01976fe1b4bda4fa47dc72c6f50a6dc7608b4213f5037f673f58499aaaa: Status 404 returned error can't find the container with id 9345f01976fe1b4bda4fa47dc72c6f50a6dc7608b4213f5037f673f58499aaaa Jun 12 17:49:42.586126 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.586091 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:49:42.586263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.586167 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:49:42.586263 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:42.586199 2544 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jun 12 17:49:43.394619 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:43.394576 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" event={"ID":"2cc352ef-2b02-4819-a396-114c4245490d","Type":"ContainerStarted","Data":"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c"} Jun 12 17:49:43.395019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:43.394629 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" event={"ID":"2cc352ef-2b02-4819-a396-114c4245490d","Type":"ContainerStarted","Data":"9345f01976fe1b4bda4fa47dc72c6f50a6dc7608b4213f5037f673f58499aaaa"} Jun 12 17:49:43.412665 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:43.412600 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" podStartSLOduration=1.4125642680000001 podStartE2EDuration="1.412564268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-06-12 17:49:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:49:43.411528317 +0000 UTC m=+2300.284176425" watchObservedRunningTime="2026-06-12 17:49:43.412564268 +0000 UTC m=+2300.285212377" Jun 12 17:49:43.432118 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:43.432086 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:43.437212 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:43.437184 2544 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:44.399643 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:44.399602 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:44.400799 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:44.400780 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:49.644796 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:49.644754 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:49.645415 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:49.644990 2544 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" podUID="2cc352ef-2b02-4819-a396-114c4245490d" containerName="istio-proxy" containerID="cri-o://68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c" gracePeriod=30 Jun 12 17:49:54.905131 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.905046 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:54.960568 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960530 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960568 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960575 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9gwq\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960614 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960638 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960664 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960684 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960708 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.960822 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960743 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.961125 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.960844 2544 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert\") pod \"2cc352ef-2b02-4819-a396-114c4245490d\" (UID: \"2cc352ef-2b02-4819-a396-114c4245490d\") " Jun 12 17:49:54.961125 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961035 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:49:54.961381 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961337 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:49:54.961717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961397 2544 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-certs\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:54.961717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961441 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 12 17:49:54.961717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961568 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:49:54.961717 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.961587 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data" (OuterVolumeSpecName: "istio-data") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:49:54.963409 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.963381 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq" (OuterVolumeSpecName: "kube-api-access-p9gwq") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "kube-api-access-p9gwq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:49:54.963545 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.963426 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jun 12 17:49:54.963610 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.963557 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token" (OuterVolumeSpecName: "istio-token") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 12 17:49:54.963692 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:54.963656 2544 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2cc352ef-2b02-4819-a396-114c4245490d" (UID: "2cc352ef-2b02-4819-a396-114c4245490d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 12 17:49:55.062586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062544 2544 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2cc352ef-2b02-4819-a396-114c4245490d-istiod-ca-cert\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062578 2544 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2cc352ef-2b02-4819-a396-114c4245490d-istio-podinfo\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062587 2544 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9gwq\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-kube-api-access-p9gwq\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062586 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062597 2544 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2cc352ef-2b02-4819-a396-114c4245490d-istio-token\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062868 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062605 2544 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-credential-socket\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062868 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062613 2544 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-data\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062868 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062621 2544 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-istio-envoy\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.062868 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.062628 2544 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2cc352ef-2b02-4819-a396-114c4245490d-workload-socket\") on node \"ip-10-0-128-121.ec2.internal\" DevicePath \"\"" Jun 12 17:49:55.454525 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.454486 2544 generic.go:356] "Generic (PLEG): container finished" podID="2cc352ef-2b02-4819-a396-114c4245490d" containerID="68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c" exitCode=0 Jun 12 17:49:55.454723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.454540 2544 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" Jun 12 17:49:55.454723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.454582 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" event={"ID":"2cc352ef-2b02-4819-a396-114c4245490d","Type":"ContainerDied","Data":"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c"} Jun 12 17:49:55.454723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.454636 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v" event={"ID":"2cc352ef-2b02-4819-a396-114c4245490d","Type":"ContainerDied","Data":"9345f01976fe1b4bda4fa47dc72c6f50a6dc7608b4213f5037f673f58499aaaa"} Jun 12 17:49:55.454723 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.454661 2544 scope.go:117] "RemoveContainer" containerID="68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c" Jun 12 17:49:55.464367 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.464349 2544 scope.go:117] "RemoveContainer" containerID="68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c" Jun 12 17:49:55.464658 ip-10-0-128-121 kubenswrapper[2544]: E0612 17:49:55.464641 2544 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c\": container with ID starting with 68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c not found: ID does not exist" containerID="68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c" Jun 12 17:49:55.464702 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.464668 2544 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c"} err="failed to get container status \"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c\": rpc error: code = NotFound desc = could not find container \"68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c\": container with ID starting with 68cdd717a8177a586b0e3bb47a09f447660fec8b8f46ab2ede0c79a37c37486c not found: ID does not exist" Jun 12 17:49:55.476570 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.476542 2544 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:55.480082 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.480058 2544 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-622ae149-openshift-default-7fdc56b798-v6p5v"] Jun 12 17:49:55.602871 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:49:55.602835 2544 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cc352ef-2b02-4819-a396-114c4245490d" path="/var/lib/kubelet/pods/2cc352ef-2b02-4819-a396-114c4245490d/volumes" Jun 12 17:50:07.968941 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:07.968898 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7cbdbb8cc9-bnktd_ea2a518d-f699-4fc0-ab32-460eff046354/nginx/0.log" Jun 12 17:50:08.440843 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:08.440812 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-86b7d85b46-rgdwt_d6bef020-e5d7-4821-bbf6-546fdaaa2f10/maas-api/0.log" Jun 12 17:50:09.288489 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:09.288460 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86876fc-bpbwc_cfaa229a-3910-42ed-946d-a6b5c89fad58/manager/0.log" Jun 12 17:50:09.299150 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:09.299124 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86876fc-bpbwc_cfaa229a-3910-42ed-946d-a6b5c89fad58/copy-manifests/0.log" Jun 12 17:50:11.074888 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:11.074800 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cscdg_27787e06-1ed6-49b8-8376-1a1871597bc5/manager/0.log" Jun 12 17:50:11.305514 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:11.305481 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-82dj6_29fb7620-3600-40c3-91ca-03d6e371d405/registry-server/0.log" Jun 12 17:50:11.575560 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:11.575529 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6qbxg_3f862d37-568a-4cd8-95d2-80bd84e23b50/limitador/0.log" Jun 12 17:50:11.696020 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:11.695976 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jq55v_7b80fc63-1eae-4910-a932-ce76ea86d36b/manager/0.log" Jun 12 17:50:12.042956 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:12.042915 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bcjh74t_22a13ca2-c060-42f8-bd5c-466ae88f3508/istio-proxy/0.log" Jun 12 17:50:12.506000 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:12.505962 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/1.log" Jun 12 17:50:12.867471 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:12.867440 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:50:13.222287 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.222201 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj_4d6c85ea-9447-40fc-9674-72a4ad1556eb/main/0.log" Jun 12 17:50:13.230268 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.230236 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-f5c55bdc-ptzwj_4d6c85ea-9447-40fc-9674-72a4ad1556eb/storage-initializer/0.log" Jun 12 17:50:13.343246 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.343215 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-b97vm_185b0c97-21ce-4f48-bfca-54aa42ae72ae/storage-initializer/0.log" Jun 12 17:50:13.352644 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.352618 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5cdff5474d-b97vm_185b0c97-21ce-4f48-bfca-54aa42ae72ae/main/0.log" Jun 12 17:50:13.585813 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.585724 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm_d941e7ee-9874-48f3-b497-e21cc223eb72/main/0.log" Jun 12 17:50:13.593954 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.593931 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-5b477c27xgm_d941e7ee-9874-48f3-b497-e21cc223eb72/storage-initializer/0.log" Jun 12 17:50:13.701888 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.701855 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-6297k_814c4d44-5f5a-446f-b9f6-81eb8db54275/storage-initializer/0.log" Jun 12 17:50:13.710073 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:13.710046 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-6f969c879b-6297k_814c4d44-5f5a-446f-b9f6-81eb8db54275/main/0.log" Jun 12 17:50:20.615217 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:20.615182 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p25tr_40d265f2-894f-43a3-8643-36b1157cf081/global-pull-secret-syncer/0.log" Jun 12 17:50:20.803817 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:20.803789 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-hp4d4_2acee30e-f461-4ee5-9a33-03ee6e876d3e/konnectivity-agent/0.log" Jun 12 17:50:20.868607 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:20.868522 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-121.ec2.internal_d7b965d585d2a6f0e3ab4a9c481d7616/haproxy/0.log" Jun 12 17:50:25.829001 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:25.828961 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-cscdg_27787e06-1ed6-49b8-8376-1a1871597bc5/manager/0.log" Jun 12 17:50:25.915845 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:25.915813 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-82dj6_29fb7620-3600-40c3-91ca-03d6e371d405/registry-server/0.log" Jun 12 17:50:26.074185 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:26.074085 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-6qbxg_3f862d37-568a-4cd8-95d2-80bd84e23b50/limitador/0.log" Jun 12 17:50:26.170672 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:26.170636 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-jq55v_7b80fc63-1eae-4910-a932-ce76ea86d36b/manager/0.log" Jun 12 17:50:27.874376 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:27.874342 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5b5ff785fc-jmv2h_8b28f84f-ec7e-4077-87f1-7ebb168b8e7f/metrics-server/0.log" Jun 12 17:50:27.940040 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:27.939994 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l9jmw_1b1e37dc-9346-42f0-b844-0eca9a1a1bd2/node-exporter/0.log" Jun 12 17:50:27.961711 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:27.961687 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l9jmw_1b1e37dc-9346-42f0-b844-0eca9a1a1bd2/kube-rbac-proxy/0.log" Jun 12 17:50:27.982970 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:27.982942 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-l9jmw_1b1e37dc-9346-42f0-b844-0eca9a1a1bd2/init-textfile/0.log" Jun 12 17:50:28.273001 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.272972 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/prometheus/0.log" Jun 12 17:50:28.293910 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.293879 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/config-reloader/0.log" Jun 12 17:50:28.316901 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.316870 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/thanos-sidecar/0.log" Jun 12 17:50:28.356656 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.356620 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/kube-rbac-proxy-web/0.log" Jun 12 17:50:28.378351 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.378322 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/kube-rbac-proxy/0.log" Jun 12 17:50:28.402465 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.402436 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/kube-rbac-proxy-thanos/0.log" Jun 12 17:50:28.425782 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.425757 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_a881a870-c24a-4f10-a2ac-cdd15960c697/init-config-reloader/0.log" Jun 12 17:50:28.627511 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.627482 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/thanos-query/0.log" Jun 12 17:50:28.666009 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.665979 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/kube-rbac-proxy-web/0.log" Jun 12 17:50:28.695817 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.695772 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/kube-rbac-proxy/0.log" Jun 12 17:50:28.717374 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.717345 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/prom-label-proxy/0.log" Jun 12 17:50:28.740420 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.740383 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/kube-rbac-proxy-rules/0.log" Jun 12 17:50:28.763625 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:28.763594 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-c65cbb6b-gwbb5_ecc155ca-673e-48a4-801b-dd081e8a099c/kube-rbac-proxy-metrics/0.log" Jun 12 17:50:32.197386 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.197343 2544 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll"] Jun 12 17:50:32.197771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.197734 2544 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cc352ef-2b02-4819-a396-114c4245490d" containerName="istio-proxy" Jun 12 17:50:32.197771 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.197745 2544 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cc352ef-2b02-4819-a396-114c4245490d" containerName="istio-proxy" Jun 12 17:50:32.197864 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.197811 2544 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cc352ef-2b02-4819-a396-114c4245490d" containerName="istio-proxy" Jun 12 17:50:32.202138 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.202120 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.204441 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.204414 2544 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-smtkg\"/\"default-dockercfg-tkgz7\"" Jun 12 17:50:32.205005 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.204957 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-smtkg\"/\"openshift-service-ca.crt\"" Jun 12 17:50:32.205005 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.204985 2544 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-smtkg\"/\"kube-root-ca.crt\"" Jun 12 17:50:32.210124 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.210096 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll"] Jun 12 17:50:32.220185 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.220144 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-proc\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.220370 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.220202 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-podres\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.220370 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.220275 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-sys\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.220526 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.220398 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bbc9\" (UniqueName: \"kubernetes.io/projected/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-kube-api-access-9bbc9\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.220526 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.220439 2544 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-lib-modules\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.321907 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.321863 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-proc\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.321917 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-podres\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.321959 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-sys\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322006 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-proc\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322009 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bbc9\" (UniqueName: \"kubernetes.io/projected/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-kube-api-access-9bbc9\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322072 2544 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-lib-modules\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322097 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322079 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-sys\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322372 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322098 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-podres\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.322372 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.322191 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-lib-modules\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.330030 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.330007 2544 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bbc9\" (UniqueName: \"kubernetes.io/projected/48006b84-9338-4181-b6a8-f1c1c6bcd7ec-kube-api-access-9bbc9\") pod \"perf-node-gather-daemonset-bd8ll\" (UID: \"48006b84-9338-4181-b6a8-f1c1c6bcd7ec\") " pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.513019 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.512919 2544 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:32.651835 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:32.651656 2544 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll"] Jun 12 17:50:32.654551 ip-10-0-128-121 kubenswrapper[2544]: W0612 17:50:32.654520 2544 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod48006b84_9338_4181_b6a8_f1c1c6bcd7ec.slice/crio-6e8df780c4c3c82f55aae4c9231c7d91628ac8e275170277dd515798f9c654b0 WatchSource:0}: Error finding container 6e8df780c4c3c82f55aae4c9231c7d91628ac8e275170277dd515798f9c654b0: Status 404 returned error can't find the container with id 6e8df780c4c3c82f55aae4c9231c7d91628ac8e275170277dd515798f9c654b0 Jun 12 17:50:33.423563 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.423535 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpt28_c79afb0f-1937-47de-aeb0-99600d87bd6c/dns/0.log" Jun 12 17:50:33.447440 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.447410 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-rpt28_c79afb0f-1937-47de-aeb0-99600d87bd6c/kube-rbac-proxy/0.log" Jun 12 17:50:33.496398 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.496369 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p25h9_e159f176-bd80-4a5a-93c2-abcbd609ec4f/dns-node-resolver/0.log" Jun 12 17:50:33.639926 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.639887 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" event={"ID":"48006b84-9338-4181-b6a8-f1c1c6bcd7ec","Type":"ContainerStarted","Data":"42cae3cf4ca503da176da8021c63a23d861b045e620ac3b07215425e8fa023bf"} Jun 12 17:50:33.640091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.639947 2544 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" event={"ID":"48006b84-9338-4181-b6a8-f1c1c6bcd7ec","Type":"ContainerStarted","Data":"6e8df780c4c3c82f55aae4c9231c7d91628ac8e275170277dd515798f9c654b0"} Jun 12 17:50:33.640091 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.640057 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:33.655171 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:33.655121 2544 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" podStartSLOduration=1.655105889 podStartE2EDuration="1.655105889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-06-12 17:50:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-06-12 17:50:33.653863845 +0000 UTC m=+2350.526511981" watchObservedRunningTime="2026-06-12 17:50:33.655105889 +0000 UTC m=+2350.527753997" Jun 12 17:50:34.062237 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:34.062199 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5df4d4978c-5t5q2_2bd6c12a-724c-4049-b22f-771b7a07b33d/registry/0.log" Jun 12 17:50:34.086516 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:34.086485 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6hgck_028bb6d9-6629-443c-a31d-83abf2688151/node-ca/0.log" Jun 12 17:50:35.054025 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:35.053990 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bcjh74t_22a13ca2-c060-42f8-bd5c-466ae88f3508/istio-proxy/0.log" Jun 12 17:50:35.205432 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:35.205399 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/0.log" Jun 12 17:50:35.267679 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:35.267644 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/1.log" Jun 12 17:50:35.429766 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:35.429734 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:50:36.112641 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:36.112614 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-nxlld_665d5681-1fe2-400a-8c15-53e4817e0327/serve-healthcheck-canary/0.log" Jun 12 17:50:36.628649 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:36.628620 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5sgrm_14df4b50-7be7-4b74-aec5-e6fd2430074f/kube-rbac-proxy/0.log" Jun 12 17:50:36.651881 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:36.651851 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5sgrm_14df4b50-7be7-4b74-aec5-e6fd2430074f/exporter/0.log" Jun 12 17:50:36.675438 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:36.675405 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5sgrm_14df4b50-7be7-4b74-aec5-e6fd2430074f/extractor/0.log" Jun 12 17:50:39.032777 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:39.032746 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_dashboard-redirect-7cbdbb8cc9-bnktd_ea2a518d-f699-4fc0-ab32-460eff046354/nginx/0.log" Jun 12 17:50:39.229634 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:39.229601 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-86b7d85b46-rgdwt_d6bef020-e5d7-4821-bbf6-546fdaaa2f10/maas-api/0.log" Jun 12 17:50:39.524993 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:39.524962 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86876fc-bpbwc_cfaa229a-3910-42ed-946d-a6b5c89fad58/manager/0.log" Jun 12 17:50:39.545358 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:39.545332 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-d86876fc-bpbwc_cfaa229a-3910-42ed-946d-a6b5c89fad58/copy-manifests/0.log" Jun 12 17:50:39.655401 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:39.655375 2544 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-smtkg/perf-node-gather-daemonset-bd8ll" Jun 12 17:50:41.393040 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:41.393003 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-574bbc8b44-lb7k4_7a3a0cde-4422-4306-9105-223d6499ed18/manager/0.log" Jun 12 17:50:41.446322 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:41.446244 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-2xjp5_95df3d51-8124-44b2-9cf2-fe6f7c773261/openshift-lws-operator/0.log" Jun 12 17:50:44.912253 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:44.912216 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bcjh74t_22a13ca2-c060-42f8-bd5c-466ae88f3508/istio-proxy/0.log" Jun 12 17:50:45.038980 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:45.038945 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/0.log" Jun 12 17:50:45.103500 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:45.103463 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/1.log" Jun 12 17:50:45.241547 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:45.241445 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:50:46.433906 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:46.433869 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bcjh74t_22a13ca2-c060-42f8-bd5c-466ae88f3508/istio-proxy/0.log" Jun 12 17:50:46.560261 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:46.560221 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/0.log" Jun 12 17:50:46.627848 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:46.627813 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/1.log" Jun 12 17:50:46.767957 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:46.767852 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:50:48.396964 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.396935 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-5b556588bcjh74t_22a13ca2-c060-42f8-bd5c-466ae88f3508/istio-proxy/0.log" Jun 12 17:50:48.402337 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.402282 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/kube-multus-additional-cni-plugins/0.log" Jun 12 17:50:48.426382 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.426357 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/egress-router-binary-copy/0.log" Jun 12 17:50:48.449321 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.449274 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/cni-plugins/0.log" Jun 12 17:50:48.471851 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.471827 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/bond-cni-plugin/0.log" Jun 12 17:50:48.495676 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.495639 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/routeoverride-cni/0.log" Jun 12 17:50:48.509738 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.509687 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/0.log" Jun 12 17:50:48.517203 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.517177 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/whereabouts-cni-bincopy/0.log" Jun 12 17:50:48.540976 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.540939 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-dzhv9_3aba6f38-a65a-43a8-86db-43d96ad371ee/whereabouts-cni/0.log" Jun 12 17:50:48.578473 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.578440 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-8559cd5744-6vdsv_d11e658c-3eba-42e0-8aad-e3f82505bab6/istio-proxy/1.log" Jun 12 17:50:48.712050 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.711939 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5859ddd786-h5s8d_5d2dc626-325e-41ff-b854-bf4c9ce12ea7/router/0.log" Jun 12 17:50:48.833878 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.833848 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-lfd6c_fadb0b7d-f055-467b-a67a-f034a8cfee0a/kube-multus/0.log" Jun 12 17:50:48.951626 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.951598 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ws7n9_2fe3599f-2dd4-4b8e-a4c9-573c9c893623/network-metrics-daemon/0.log" Jun 12 17:50:48.971765 ip-10-0-128-121 kubenswrapper[2544]: I0612 17:50:48.971688 2544 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-ws7n9_2fe3599f-2dd4-4b8e-a4c9-573c9c893623/kube-rbac-proxy/0.log"